Haji Rufai

Software Engineer

Upwork (formerly Elance-oDesk)•  November 2023 - Present

As a seasoned Software, Cloud, Data & Analytics Engineer on Upwork, I bring a diverse skill set and a proven track record of delivering exceptional results to clients worldwide.

Data Engineer

ExploreAI•  July 2022 - July 2023

- Developed the project architecture, designing an efficient and scalable system to handle data extraction, processing and loading to a database. - Utilized Airflow, an open-source platform, to automate various data workflows and batch data pipelines. - Worked with RDS (Relational Database Service) to manage and optimize database operations, ensuring data integrity and performance. - Actively participated in writing the white paper for the project, outlining the objectives, methodologies, and findings of our work. - Collaborated with a multidisciplinary team to gather requirements, identify pain points, and propose data engineering solutions. - Implemented data cleaning and transformation processes, ensuring data quality and consistency for downstream analysis. - Conducted performance tuning and optimization of the data pipeline, improving overall system efficiency and reducing processing time. - Contributed to the documentation of the codebase facilitating knowledge sharing and future maintenance.

Wizeline Academy

Data Engineering, Advanced Technical Diploma•  August 2022 - October 2023

Utilized Google Cloud to execute various data engineering tasks and projects. - Proficiently deployed Apache Airflow in Kubernetes clusters using Helm charts. - Processed and transformed data efficiently with DataProc and Apache Spark. - Automated the deployment of cloud resources on Google Cloud using Terraform. - Designed data pipelines to seamlessly load data from PostgreSQL Cloud SQL into Google Cloud Data Warehouse, leveraging Google BigQuery for analysis. - Implemented best practices for data storage management within Google Cloud, optimizing data retrieval and storage costs. - Employed Google Cloud Composer (Apache Airflow managed service) for workflow orchestration and automation, streamlining complex processes and ensuring task execution efficiency.