Job Description
Key Responsibilities
Design, develop, and maintain scalable ETL/ELT pipelines to ingest, process, and transform structured and unstructured data.
Build and optimize batch and real-time data pipelines using AWS services such as Redshift, S3, and Glue.
Develop robust data integration workflows to ensure high data quality, reliability, and performance.
Optimize data storage, transformation, and retrieval processes for large-scale datasets.
Collaborate with data analysts, data scientists, and business stakeholders to understand data requirements and deliver scalable solutions.
Monitor and troubleshoot data pipelines, ensuring timely issue resolution and system stability.
Implement best practices for data governance, security, and compliance.
Automate data workflows and support CI/CD deployment processes for data engineering solutions.
Maintain documentation for data architecture, pipelines, and operational processes.
Required Skills & Qualifications
Strong experience in Data Engineering and ETL/ELT development.
Hands-on experience with AWS cloud services including Redshift, S3, and Glue.
Experience building batch and real-time data processing pipelines.
Proficiency in SQL and data modeling concepts.
Strong programming skills in Python, PySpark, or Scala.
Experience with workflow orchestration tools such as Airflow or similar platforms.
Knowledge of big data technologies and distributed processing frameworks is an added advantage.
Understanding of data warehousing concepts and cloud-native architectures.
