Akash singh

Data Engineer

Cognizant•  July 2021 - Present•  Noida

Implemented new column fields and tables, modified Python scripts, and enhanced load functionality based on business logic. Developed delta-type table logic on Databricks using Spark, significantly reducing task completion time to minutes. Designed an end-to-end Azure pipeline connecting to MSSQL and Oracle (OLTP) environments, transferring data to Azure Blob Storage and loading it into Snowflake. Led the modernization of Informatica workflows and Teradata to AzureADF, Python, and Snowflake, enhancing performance and scalability. Conducted a POC on internal versus external staging area load performance to Snowflake using the COPY INTO command. Processed large data files in Snowflake by dividing them into smaller chunks to reduce processing cost and time. Implemented retry logic in Python scripts to handle and fix connection timeout errors in Snowflake.