Deepak Atrey

Senior Data Engineer

Morgan Stanley•  September 2017 - Present

➢Responsible to extracting and combining data from various heterogeneous data sources for maintaining Data Warehousing. ➢Working on creating data pipeline to load data in DataLake.(Hadoop HDFS, hive and Spark processing) ➢Responsible for data wrangling in Python data frames doing things such as filtering, sorting, joining, and analyzing data sets ➢Responsible to design new data pipeline process to handle big volume of data. ➢Responsible to interacting with Business user to gather requirement and develop new Datasets in DB2. ➢Automated process to on board new product in project using Shell scripting and Python scripting. ➢Responsible to perform tuning application and SQL query performance. ➢Develop data processing engine using pySpark and load them to Hive for BI use. ➢ Implemented Dev_Ops model in our project using CICD for integration and deployment to reduce manual intervention during migration and deployment.