Senior Data Engineer (AWS, Python/Java)

About the Role

We are looking for an experienced Data Engineer with strong expertise in AWS data engineering and Big Data technologies to design, develop and maintain scalable data pipelines that support analytics and machine learning initiatives.

Roles & Responsibilities

• Design, develop and maintain scalable data pipelines and ETL processes using AWS services such as S3, Glue, EMR, Lambda and Redshift.
• Collaborate with Data Scientists and Analysts to understand data requirements and deliver solutions for analytics and machine learning use cases.
• Optimize data storage and retrieval mechanisms for performance, reliability and cost efficiency.
• Implement data governance, security and compliance best practices to maintain data integrity.
• Troubleshoot and resolve data pipeline issues while ensuring proactive monitoring and reliability.
• Design and optimize large-scale data processing workflows using Big Data and distributed computing technologies.
• Stay updated with emerging data engineering technologies and recommend solutions to improve engineering capabilities.

Ideal Candidate

• 6+ years of experience in Data Engineering with strong expertise in designing and maintaining scalable data pipelines and ETL processes.
• 3+ years of hands-on AWS data engineering experience, particularly with S3, AWS Glue, EMR, Lambda and Redshift.
• Strong programming skills in Python, Java or Scala with hands-on experience developing data processing and pipeline solutions.
• Strong expertise in SQL, data warehousing and database technologies, with experience across both SQL and NoSQL databases.
• Hands-on experience with Big Data technologies and distributed data processing for large-scale data pipelines.
• Strong experience in pipeline troubleshooting, monitoring, data quality, governance and security.
• B.Tech / B.E. / M.Tech is mandatory.
• Immediate joiners or candidates who can join within 2 weeks are highly preferred.
• Experience with Apache Airflow, Docker and Kubernetes will be an added advantage.