Hare kishan s

Associate consultant

Capgemini•  January 2025 - Present

Building pipelines for movement of data from GCS to BQ through Airflow composer. Creating alerting policy for the pipelines using google cloud monitoring. Re-engineered the Snowflake codebase, seamlessly integrating new instances. Orchestrated the end-to-end flow design for file movement, from on-prem to GCS Cloud, resulting in a 30% reduction in data transfer errors. Design and implemented the file watcher and file validation methodologies for HPOS to GCS migration. Developed the Discovery services using rest API for Teradata, FTP, SFTP, Snowflake, Bigquery, synapse. Created a metadata-driven architecture using MongoDB that includes the discovery plane (metadata visualization). Re-engineered AZURE migration code base, helps in reducing lines of code by 40% and creation of API for different feature functionalities. Developed ETL conversion scripts for stored procedures, Snowflake DDL, and HIVE DDL. It reduces the manual conversion by 50%. Optimizing the Discovery service by implementing threading technique which increases the efficiency by 80%