Data Engineer

Role: Senior AWS Data Engineer

Experience: 8+ Years

Location: Pune

We are looking for an experienced Senior AWS Data Engineer to join our data engineering team and work on scalable cloud data platforms, real-time data processing, and high-volume data pipelines.

The ideal candidate will have strong hands-on expertise in AWS Data Engineering, PySpark, Python, AWS Glue, Amazon EMR, Apache Kafka, Spark Streaming, Hudi, Iceberg, Docker, ECS, and Terraform.

Key Responsibilities

• Design, develop, and maintain scalable AWS data pipelines and data platforms.
• Build high-performance ETL workflows using AWS Glue, Glue ETL, and Glue Data Catalog.
• Develop distributed data processing solutions using Python, PySpark, and Apache Spark.
• Build real-time streaming pipelines using Apache Kafka and Spark Streaming.
• Work with large-scale data lake environments using Apache Hudi and Apache Iceberg.
• Develop and optimize distributed processing workloads using Amazon EMR.
• Design reliable and fault-tolerant data pipelines for high-volume, real-time data.
• Use Docker and Amazon ECS for application deployment and containerization.
• Implement infrastructure automation using Terraform.
• Apply strong software engineering and OOP principles to build maintainable data solutions.
• Collaborate with engineering, architecture, and business teams to deliver production-ready data platforms.

Required Skills

• 8+ years of experience in Data Engineering.
• Strong hands-on experience with AWS Data Engineering.
• Expert-level experience with Python and PySpark.
• Strong experience with AWS Glue, Glue ETL, and Glue Data Catalog.
• Hands-on experience with Amazon EMR.
• Strong knowledge of Apache Kafka and Spark Streaming.
• Experience with Apache Hudi and/or Apache Iceberg.
• Experience with Docker and Amazon ECS.
• Good understanding of Terraform and infrastructure automation.
• Strong understanding of OOP/OOPS and software engineering practices.
• Experience working with high-volume and real-time data processing environments.