Akshay Krishna

Data Engineer

Vodafone•  July 2018 - August 2021

• Vodafone Star Employee of the Month May 2020: Led and devised Hadoop and Streamsets data pipeline cluster, also developed format changing and KPI dashboard PySpark applications for recovering lost data from Splunk. • Estimated the real-time message counts with 100% precision in Kafka and displayed topic wise time-based statistics in Splunk Dashboard. • Extracted data from Splunk dashboards and created automated mail service to users from the Splunk data operating Python and Spark. • Assembled and maintained data pipeline management cluster and engineered data pipelines with transformations from diverse sources diminishing the data size by 40 times into AVRO, Parquet and JSON formats and loaded into Hadoop. • Performed timeseries analysis on data pipelines for runtime statistics and migrated InfluxDB time series and MySQL database to different nodes for better performance of data pipelines and reducing cluster resources usage by 60%. • Diagnosed the resource usage for long running jobs and designed automated Python script for pipeline statuses using the underlying metadata from SQL database and data pipeline REST API. • Administered application servers handling incoming traffic and transformed real time data using Logstash and Filebeat with 0% loss of the data.