Basudev Chhotaray

Data Engineer

Blackbuck Insights pvt ltd•  January 2021 - Present

Blackbuck Insights Pvt. Ltd. (18/01/2021 – Present) Projects: 1. TransUnion | 01/11/2022 - Present − Spearheaded the development of a credit information processing and analytics platform, emphasizing Apache Spark, Python, and DevOps practices. Achievements: • Designed and optimized ETL processes for large credit datasets using Apache Spark and Python. • Implemented DevOps strategies with Ansible, Jenkins, and Terraform for automation and infrastructure management. • Collaborated with data scientists, utilizing Apache Airflow for workflow automation and insights derivation. • Evaluated and implemented new technologies, including Docker for containerization, to enhance data processing efficiency. • Led efforts to enhance data security measures and ensure compliance with industry standards. 2. CIOX Health| 10/08/2021 - 16/09/2022 − Led the development of an integrated health data processing platform, leveraging AWS, Apache Spark, Python, and Apache Airflow. Achievements: • Engineered robust end-to-end ETL processes utilizing Apache Spark for diverse health data sources. • Implemented advanced data quality checks, enhancing the integrity and accuracy of processed health data. • Integrated Apache Airflow for complex workflow orchestration, automating ETL processes. • Utilized DevOps practices, including Ansible and Jenkins, for deployment automation. • Played a key role in integrating new health data sources using Snowflake as the primary data warehouse.