Swapnali Kore

Solution Developer

TATA TECHNOLOGIES LTD.•  December 2021 - Present•  Pune

Managed ETL operations between on-premises and Azure services (Azure SQL, Blob Storage, Data Lake Gen2) using Azure Data Factory V2. Developed and optimized data extraction and transformation processes from SQL servers, APIs, flat files, and SharePoint, ensuring data accuracy and efficiency. Implemented incremental loads strategies for multiple tables using a single pipeline, enhancing data refresh efficiency and reducing processing time. Utilized advanced Azure Data Factory features (Copy Data, Stored Procedures, Filters) to streamline data workflows and resolve pipeline failures. Worked extensively with Azure DevOps to migrate pipelines through development, pre-production, and production environments, ensuring smooth deployment and version control. Designed and executed ETL workflows with Matillion, loading data into Snowflake databases and enhancing analytics capabilities. Developed and optimized data queries for dashboards using DBeaver, implementing ETL processes orchestrated with Apache Airflow. Authored stored procedures, incorporating joins and cursors to meet specific client requirements, resulting in improved data integration and performance. Worked with Azure ML Studio to build pipelines using its components, facilitating machine learning model development and deployment. Collaborated with cross-functional teams to gather requirements and provide data solutions aligned with business objectives.