shailendra mishra

Software Engineer

Multiplan•  January 2022 - Present

• Design and develop data lake solutions for enterprise , end to end ETL solutions from various source systems to staging tables, transform the data and generate BI reports for Vendors. • Creating databricks notebook/Spark and Azure data factory pipeline/dataflow for different ETL/ELT solution using Python . • Expose datalake data to business using Snowflake • Using Informatica as ETL for all on Premises ETL solutions • Created PL/SQL Packages,PL/SQL blocks, Procedures, Informatica mappings, workflows, sessions/tasks. • Develop and manage legacy reporting system with the help of PL/SQL Code . • Parameterize workflow/sessions/mappings to pass parameters/variables through shell scripts and Tidal jobs. • Deployed Informatica objects from Dev to Test servers • Designed and developed claim processing using Informatica PowerCenter. • Designed and developed new solutions in Palantir Foundry using SQL queries- joins, CTE, aggregate functions, group by, UNION etc and generated reports as per business requirements. • Code review of other team member’s codes. • Production support and On-call duties, monitoring jobs and resolving production issues during weekends and non-business hours. • Experience in data governance to follow healthcare industry standard HIPAA compliance for claims and Provider data. • Used different tool like IDQ for data quality and integrity of data for Provider and client domain data • Use different tools TDM/Azure for data governance/masking and data subletting. • Strong experience with Big Data technologies, ETL process, data analysis and complex SQL • Experience on different data quality problem and prepare data which is best fit for Enterprise. • Experience in Event hub to processing file into data lake. • Part of Team which Design and Build Azure Data platform from scratch • Perform POC on Azure data science use cases • Create reusable UDF in python for generic use . • Used Spark API component like Dataframe , Datasets ,RDD using python • Worked on Spark Streaming use cases with python as programming language to develop solutions

Data Enginner

Synapsis•  June 2021 - December 2021

• Design and develop data lake solutions for enterprise , end to end ETL solutions from various source systems to staging tables, transform the data and generate BI reports for Vendors. • Creating databricks notebook/Spark and Azure data factory pipeline/dataflow for different ETL/ELT solution using Python . • Expose datalake data to business using Snowflake • Strong experience with Big Data technologies, ETL process, data analysis and complex SQL • Experience on different data quality problem and prepare data which is best fit for Enterprise. • Experience in Event hub to processing file into datalake using Azure Functions. • Part of Team which Design and Build Azure Data platform from scratch • Perform POC on Azure data science use cases • Create reusable UDF in python for generic use . • Used Spark API component like Dataframe , Datasets ,RDD using python • Worked on Spark Streaming use cases with python as programming language to develop solutions

Manager

Cognizant Technology Solutions•  August 2014 - June 2021

• Design and develop data lake solutions for enterprise , end to end ETL solutions from various source systems to staging tables, transform the data and generate BI reports for Vendors. • Creating databricks notebook/Spark and Azure data factory pipeline/dataflow for different ETL/ELT solution using Python . • Expose datalake data to business using HDInsights or Polybase • Using Informatica as ETL for all on Premises ETL solutions • Created PL/SQL Packages,PL/SQL blocks, Procedures, Informatica mappings, workflows, sessions/tasks. • Develop and manage legacy reporting system with the help of PL/SQL Code . • Parameterize workflow/sessions/mappings to pass parameters/variables through shell scripts and Tidal jobs. • Deployed Informatica objects from Dev to Test servers • Designed and developed claim processing using Informatica PowerCenter. • Designed and developed new solutions in Palantir Foundry using SQL queries- joins, CTE, aggregate functions, group by, UNION etc and generated reports as per business requirements. • Code review of other team member’s codes. • Production support and On-call duties, monitoring jobs and resolving production issues during weekends and non-business hours. • Experience in data governance to follow healthcare industry standard HIPAA compliance for claims and Provider data. • Used different tool like IDQ for data quality and integrity of data for Provider and client domain data • Use different tools TDM/Azure for data governance/masking and data subletting. • Strong experience with Big Data technologies, ETL process, data analysis and complex SQL • Experience on different data quality problem and prepare data which is best fit for Enterprise. • Experience in Event hub to processing file into data lake. • Part of Team which Design and Build Azure Data platform from scratch • Perform POC on Azure data science use cases • Create reusable UDF in python for generic use . • Used Spark API component like Dataframe , Datasets ,RDD using python • Worked on Spark Streaming use cases with python as programming language to develop solutions

IT Analyst

Tata Consultancy Services•  April 2011 - August 2014

Translated business processes into Informatica mappings by using Informatica Designer which populated the data into the Analytical data warehouse with Oracle  Wrote test plans and executed the same at unit testing and also supported for system testing, volume testing and user testing.  Worked on developing Informatica Mappings, Mapplets, Sessions, Workflows and Worklets for data loads.  Performed various data cleansing and data conversion functions in various transformations.  Developed mapping spreadsheets with source to target data mapping with physical naming standards, datatypes, domain definitions, and corporate meta-data definitions.  Used different cache properties, such as dynamic cache session property to perform the update and insert into the relational table simultaneously using Lookup transformation.  Tuned performance of Informatica session for large data files by increasing block size, data cache size, sequence buffer length and target based on commit interval.  Prepared the documents required for code migration and was responsible for code deployment to QA and production.

Software Engineer

App Associates•  August 2010 - March 2011

Translated business processes into Informatica mappings by using Informatica Designer which populated the data into the Analytical data warehouse with Oracle  Wrote test plans and executed the same at unit testing and also supported for system testing, volume testing and user testing.  Worked on developing Informatica Mappings, Mapplets, Sessions, Workflows and Worklets for data loads.  Performed various data cleansing and data conversion functions in various transformations.  Developed mapping spreadsheets with source to target data mapping with physical naming standards, datatypes, domain definitions, and corporate meta-data definitions.  Used different cache properties, such as dynamic cache session property to perform the update and insert into the relational table simultaneously using Lookup transformation.  Tuned performance of Informatica session for large data files by increasing block size, data cache size, sequence buffer length and target based on commit interval.  Prepared the documents required for code migration and was responsible for code deployment to QA and production.

Senior Software Engineer

NTT Data•  May 2007 - March 2010

Translated business processes into Informatica mappings by using Informatica Designer which populated the data into the Analytical data warehouse with Oracle  Wrote test plans and executed the same at unit testing and also supported for system testing, volume testing and user testing.  Worked on developing Informatica Mappings, Mapplets, Sessions, Workflows and Worklets for data loads.  Performed various data cleansing and data conversion functions in various transformations.  Developed mapping spreadsheets with source to target data mapping with physical naming standards, datatypes, domain definitions, and corporate meta-data definitions.  Used different cache properties, such as dynamic cache session property to perform the update and insert into the relational table simultaneously using Lookup transformation.  Tuned performance of Informatica session for large data files by increasing block size, data cache size, sequence buffer length and target based on commit interval.  Prepared the documents required for code migration and was responsible for code deployment to QA and production.

Gautam Buddh Technical University, Lucknow

Electronics and Communications, B.Tech•  September 2002 - June 2006

B.Tech in Electronics and Communications