Mayank Joshi

Data Engineer

Yara International•  September 2022 - Present•  Bengaluru

Led data migration from legacy systems to new platforms, redesigning data models to meet updated business requirements and ensuring data integrity. Designed and implemented PII detection using AWS Glue and PySpark, and streamlined data ingestion with AWS DMS, Airflow, and Terraform to ensure timely and accurate data availability. Developed and optimized data models and ELT pipelines dbt, supporting analytics dashboards and providing actionable insights through efficient data transformation. Established Data Governance practices using Collibra, including a business data glossary, metadata catalog, and automated validation pipelines with Airflow and GreatExpectations. Designed geospatial data deduplication pipeline and researched Data Contracts to ensure consistent data quality and expectations across teams.

Data Analyst Intern

Careervira•  February 2022 - May 2022•  Gurugram

Developed and executed Python scripts using BeautifulSoup and Selenium for efficient web scraping, enabling the extraction of data from multiple online sources. Conducted data cleaning and processing, utilizing Python libraries such as Pandas and regular expressions, ensuring accuracy and consistency in datasets. Implemented data publishing processes in backend systems.