ram charit vyas kurra

Data Engineer

Nike•  July 2021 - Present

• Developed streaming and batch processing applications using spark to ingest data from the various sources into HDFS Data Lake. • Used Spark and Spark-SQL to read the parquet data and create the tables in hive. • Responsible in performing sort, join, aggregations, filter, and other transformations on the datasets using Spark. • Involved in converting Hive/SQL queries into Spark transformations using Spark RDD's • Involved in loading data from edge node to HDFS using shell scripting • Experience with snowflake cloud data warehouse and AWS S3 bucket for integrating data from multiple source system which includes loading parquet formatted data into snowflake table. • Developed Spark and Hadoop jobs on the EMR cluster. • Migrated an Oracle SQL ETL to run on AWS for triggering the airflow jobs. • Used Apache Airflow to build data pipelines and used various airflow operators like bash operator, Hadoop operators and python callable and branching operators.