vigneswara reddy

India

@vigneswar_reddy

Data Engineer | Python | Big Data(HDFS, Hive, Impala, Hadoop)| SQL | Pyspark Dev

Badges

Problem Solving
Python
Sql

Certifications

Work Experience

  • Senior Software Engineer

    Enquero•  July 2022 - Present

  • Senior Software Engineer

    Epsilon•  September 2019 - July 2022

     Involved in Reading data from files & Validating schema, data using pyspark data frames  Working for Clean Up Hadoop Databases by finding largest tables & unused tables from hive meta store for HDFS & AWS DB  Created Impala UDF’s for Generating Random Data with & without Predefined List of Data using core java  Based on File Triggers ActiveBatch will trigger Jobs to execute set of python jobs to clean & load data into HDFS  Involved in reading Kafka consumer messages & storing in HDFS table to store user activity in prod servers.  Working for migrating tables between databases by removing sensitive information like Prod to Dev  Working for migrating tables from encrypted zones to green zone by removing sensitive information  Working for Change requests (CR) for new/modify functionality Involved in Reading data from files & Validating schema, data using pyspark data frames  Working for Clean Up Hadoop Databases by finding largest tables & unused tables from hive meta store for HDFS & AWS DB  Created Impala UDF’s for Generating Random Data with & without Predefined List of Data using core java  Based on File Triggers ActiveBatch will trigger Jobs to execute set of python jobs to clean & load data into HDFS  Involved in reading Kafka consumer messages & storing in HDFS table to store user activity in prod servers.  Working for migrating tables between databases by removing sensitive information like Prod to Dev  Working for migrating tables from encrypted zones to green zone by removing sensitive information  Working for Change requests (CR) for new/modify functionality \n \n \t \t Skills: PySpark · Cloudera · Oracle Database · Extract, Transform, Load (ETL) · HiveQL · HDFS · Apache Impala · Python (Programming Language) · Hadoop · Apache Spark · Git

  • Software Engineer

    BT E-serv India Pvt Ltd•  February 2017 - September 2019

     As part of Data warehouse maintenance, we have moved older data from RDMS to HAAS to reduce burden on Transaction database and to improve performance.  Instead of creating DB links and giving access to other component's team , we have exported data into HDFS and shared access so that via Hue browser they access data and based on user login & group we had controlled access  Created Hive DDLs for loading CSV, XML files into stage tables and applying business transformations  Involved in Moving tables Oracle from Hadoop by using Oozie workflow with Sqoop  Involved in Design, Development & Testing as part of Oracle Discoverer 10G/11G to Oracle Business Intelligence 12C Migration  Design, Develop, Test, and modify OBIEE security using roles, privileges in Enterprise Manager (EM), Administration Console to Access security  Created star schemas in the repository. Used the foreign key link in the physical layer and the new complex joint key in the Business Model and Mapping layer to form relationships between fact and dimensional tables.  Developed and deploy BI Answer, BI Publisher Reports & dashboards properly  Merged the multiple RPD and rolled to the production & created iBots/Intelligence Agents for job- critical levels and auditable differences delivered to dashboards  Worked on Big Data Technologies like Hive, Impala, Sqoop, Oozie Languages & HDFS  Created Hive DDLs for loading CSV, XML files into stage tables and applying business transformations  As part of Data warehouse maintenance, we have moved older data from RDMS to HAAS and, we are using for sharing data to another component system  Experience in writing Oracle PL/SQL packages, Procedures, Functions, Database Triggers, Cursors, Exceptions, data modelling and advanced concepts like arrays, bulk collect and for all, Performance Tuning, and analytical functions  We have implemented Oracle Wallet for Storing Application Usernames and Passwords As part of Data warehouse maintenance, we have moved older data from RDMS to HAAS to reduce burden on Transaction database and to improve performance.  Instead of creating DB links and giving access to other component's team , we have exported data into HDFS and shared access so that via Hue browser they access data and based on user login & group we had controlled access  Created Hive DDLs for loading CSV, XML files into stage tables and applying business transformations  Involved in Moving tables Oracle from Hadoop by using Oozie workflow with Sqoop  Involved in Design, Development & Testing as part of Oracle Discoverer 10G/11G to Oracle Business Intelligence 12C Migration  Design, Develop, Test, and modify OBIEE security using roles, privileges in Enterprise Manager (EM), Administration Console to Access security  Created star schemas in the repository. Used the foreign key link in the physical layer and the new complex joint key in the Business Model and Mapping layer to form relationships between fact and dimensional tables.  Developed and deploy BI Answer, BI Publisher Reports & dashboards properly  Merged the multiple RPD and rolled to the production & created iBots/Intelligence Agents for job- critical levels and auditable differences delivered to dashboards  Worked on Big Data Technologies like Hive, Impala, Sqoop, Oozie Languages & HDFS  Created Hive DDLs for loading CSV, XML files into stage tables and applying business transformations  As part of Data warehouse maintenance, we have moved older data from RDMS to HAAS and, we are using for sharing data to another component system  Experience in writing Oracle PL/SQL packages, Procedures, Functions, Database Triggers, Cursors, Exceptions, data modelling and advanced concepts like arrays, bulk collect and for all, Performance Tuning, and analytical functions  We have implemented Oracle Wallet for Storing Application Usernames and Passwords  Skills: Oracle Database · Oracle Discoverer · Oracle Business Intelligence Suite Enterprise Edition(OBIEE) · HiveQL · HDFS · Apache Sqoop · Apache Impala · Oracle SQL Developer · Python (Programming Language) · Hadoop · PL/SQL

  • Software Developer

    Huawei Technologies India•  October 2014 - February 2017

     Experienced in Developing self-service, Inbound & Outbound Calls for Telecom & Banking Clients  Experienced in Developing Reports for end users by using Data Station  Performed the configuration, installation and maintenance of CTI application in VoiceXML(VXML), CCXML, IVR Languages  Developed Complex database objects like Stored Procedures, Functions, Packages and Triggers using SQL and PL/SQL.  Partitioned large Tables using range partition technique. Experienced in Developing self-service, Inbound & Outbound Calls for Telecom & Banking Clients  Experienced in Developing Reports for end users by using Data Station  Performed the configuration, installation and maintenance of CTI application in VoiceXML(VXML), CCXML, IVR Languages  Developed Complex database objects like Stored Procedures, Functions, Packages and Triggers using SQL and PL/SQL.  Partitioned large Tables using range partition technique. Skills: Oracle Database · Oracle SQL Developer · PL/SQL · SQL

Education

  • G Pulla Reddy Engineering College

    Electrical Engineering & Computer Science, B.Tech•  May 2010 - May 2014

    Grade: 8.2 CGPA \n \t \t Activities and societies: I worked Student co-ordinator In ELECTRONICA fest Which is held at G.P.R.E.C, KurnoolActivities and societies: I worked Student co-ordinator In ELECTRONICA fest Which is held at G.P.R.E.C, Kurnool Learning New technologies, organizing and volunteer in college fests and worked as class committee member for responsibility of the one class room. Me and my project member are the first team who worked on verilog(advanced version of VHDL) on Xilinix software in my college. For my project we got appreciation from my team leader and Head of the department(HOD) of the college.

Skills

vigneswar_reddy has not updated skills details yet.