Job Title: AWS Data Engineer
Location: Bangalore, India
Work Mode: Hybrid
About the Role
We are seeking a highly skilled Data Engineer to design, build, and optimize scalable data pipelines and lakehouse solutions on AWS. The ideal candidate will have strong experience in enterprise-scale data engineering, ETL development, and modern cloud-based data platforms. This role will be responsible for building and maintaining Medallion Architecture (Bronze, Silver, Gold) data pipelines to support analytics, reporting, and business intelligence initiatives.
Key Responsibilities
- Design, develop, and maintain AWS-based data pipelines following Medallion Architecture (Bronze, Silver, Gold).
- Build and optimize ETL/ELT workflows using AWS Glue, Spark, and Python.
- Develop and manage data ingestion pipelines using AWS DMS and Kinesis.
- Implement data transformation and enrichment processes to support analytics and reporting requirements.
- Integrate data solutions with AWS Athena, Redshift, and other downstream data consumers.
- Create and maintain datasets using Parquet and Apache Iceberg formats.
- Ensure data quality, consistency, governance, and lineage across the data platform.
- Implement monitoring, logging, and alerting mechanisms to maintain pipeline reliability.
- Collaborate with Data Architects, Data Analysts, and Business Stakeholders to understand data requirements and deliver scalable solutions.
- Participate in CI/CD implementation and automation for ETL deployment and maintenance.
- Support data cataloging, governance, and security initiatives using AWS Lake Formation and/or DataZone.
Required Skills & QualificationsTechnical Skills
- Strong hands-on experience with:
- AWS Glue
- AWS DMS (Database Migration Service)
- AWS Kinesis
- Amazon Athena
- Amazon Redshift
- Apache Spark / PySpark
- Python
- SQL
- Experience working with:
- Parquet
- Apache Iceberg
- Data Lake and Lakehouse architectures
- Knowledge of Git version control and CI/CD pipelines for data engineering workflows.
- Experience with data governance, metadata management, and cataloging tools such as AWS Lake Formation and AWS DataZone.
Experience
- 5+ years of experience in Data Engineering.
- Proven experience building enterprise-scale ETL/ELT and Lakehouse solutions.
- Experience handling large-scale data processing and analytics workloads.
- Strong understanding of data modeling, performance optimization, and data quality practices.
Preferred Qualifications
- AWS Certified Data Analytics – Specialty.
- AWS Certified Solutions Architect.
- Experience with modern data platform modernization initiatives.
- Exposure to Agile/Scrum development methodologies.
Key Performance Indicators (KPIs)
- Data pipeline reliability and uptime.
- Data freshness and timely availability.
- Adherence to ETL job SLAs.
- Data quality and accuracy metrics.
- Successful delivery of Gold-layer datasets for MVP and business consumption.
- Operational efficiency and automation of data workflows.
