Kunal Sihare

Data Engineer

Thoughtworks•  June 2024 - Present•  Gurgaon

Worked on Scala and Apache Spark within Databricks to explore and build scalable data pipelines, improving internal data processes. Assisted in data modeling and worked with Delta Lake to understand how to manage large datasets efficiently, focusing on optimizing query performance and consistency. Learned and applied pytest to write unit and integration tests for data processing workflows, ensuring robust code quality and reliability in internal projects.

Specialist Programmer

Infosys•  August 2021 - May 2024•  Pune

Collaborated with a cross-functional team to comprehend requirements and deliver solutions in an Agile environment. Engineered, developed, automated, managed, and optimized a suite of scalable data pipelines within Azure Data Factory and Azure Synapse Analytics. Utilized PySpark, Spark SQL, and Spark Core on Azure Databricks to optimize performance. Enhanced data modeling proficiency by creating and implementing dimensional data models. Executed over 10 Unit, Functional, and Integrational tests with meticulous coverage of data ingestion, integration, transformation, and quality, as well as reporting workflows.