Badges
Certifications
Work Experience
Senior Data Engineer
Deloitte•  June 2021 - Present•  United States
Built data pipelines in Azure Data Factory using Metadata Driven Framework to load on-prem data from SQL Server Databases to Azure cloud platform. Created Raw Layer in Azure Data Lake Storage and stored relational data in parquet file format for full load and incremental load using watermark strategy. Designed end-to-end flow of data from AWS S3 to Snowflake Data Warehouse through AWS Glue Workflows. Created AWS Glue Jobs using PySpark to load data from AWS RDS to staging environment in Snowflake. Loaded data from staging layer to data warehouse and implemented SCD Type 2 using stream and merge through stored procedures in Snowflake. Built Gen AI tool using OpenAI LLM to generate SQL queries from natural language, retrieve the data from the database and return the results in natural language for easy understanding.
Data Engineer Intern
Duncan Family Farms•  May 2020 - April 2021•  United States
Automated extraction of data from various data sources to Azure Synapse Data Warehouse using Microsoft Azure Cloud Platform. Created Data Pipelines using Python to execute ETL processes in Azure Data Factory. Built Denormalized Tables following Kimball Dimensional Modeling to improve performance of BI Reports. Collected data from websites through API requests and loaded the data to Data Warehouse using Python. Created Views using advanced SQL queries to produce datasets and imported data to Power BI to build reports. Delivered business-critical Customer Scorecard and other reports to facilitate data-driven decisions.
Data Engineer
Dell EMC•  February 2018 - June 2019•  India
Built Data Pipelines in Azure Data Factory to extract, transform and load data into EDW. Produced datasets from complex SQL queries, designed data models and DAX to generate Analytical Reports. Tracked status of projects, resolved impediments, prioritized change requests and defects in Azure DevOps Server.
Data Engineer
IBM•  November 2015 - February 2018•  India
Elicited requirements from Stakeholders, analyzed feasibility of the implementations and provided System Documentations. Extracted data from various sources and loaded the data into Azure Synapse Data Warehouse for data wrangling. Cleaned and transformed data as per business requirements and derived meaningful insights through BI Reporting. Analyzed dependencies with upstream & downstream applications and coordinated with cross-functional teams for day-to-day ETL progress monitoring. Designed test scripts to verify data integrity. Executed Daily Scrum Call to facilitate coordination among team members and ensure project completion within deadline.
Education
Arizona State University
Master of Science, Computer Engineering (Computer Systems)•  August 2019 - May 2021•  GPA: 4
West Bengal University of Technology
Bachelor of Technology, Electronics and Communication Engineering•  August 2011 - May 2015•  GPA: 3.9