Job Title: Senior Data Engineer (AWS)
Experience: 10 13 Years
Location: Pune / Bangalore / Hyderabad / Noida / Mumbai
Work Mode: Hybrid
Shift Timings: 11:00 AM 8:00 PM
Notice Period: Immediate Joiners Only
Job Overview
We are seeking an experienced
Senior Data Engineer with strong expertise in
Python, SQL, PySpark, and AWS data services to support enterprise-scale data processing and analytics initiatives. The ideal candidate will have deep experience in distributed data systems, performance tuning, and cloud-based data platforms, and will work closely with cross-functional teams in a hybrid working model.
Key Responsibilities
- Design, develop, and maintain scalable data processing pipelines using Python, PySpark, and Spark SQL.
- Build and optimize distributed data processing workflows on AWS platforms.
- Leverage AWS data services such as EMR, Glue, Lambda, and S3 for batch and real-time data processing.
- Design and manage data storage solutions using RDS/MySQL, Redshift, and other AWS-native databases.
- Implement effective data modeling, schema design, and schema evolution strategies.
- Perform performance tuning and optimization of Spark jobs and SQL queries.
- Monitor and troubleshoot data pipelines using AWS CloudWatch and logging frameworks.
- Manage secrets and credentials securely using AWS Secrets Manager.
- Collaborate with data architects, analysts, and stakeholders to translate business requirements into technical solutions.
- Debug complex data issues and provide root cause analysis with long-term fixes.
- Ensure data quality, reliability, and scalability across platforms.
Required Skills & Qualifications
- 1013 years of overall experience in Data Engineering.
- Strong proficiency in Python and SQL.
- Extensive hands-on experience with PySpark and Spark SQL.
- Strong experience with AWS data services, including:
- EMR
- Glue
- Lambda
- S3
- RDS / MySQL
- Redshift
- CloudWatch
- Secrets Manager
- Solid understanding of distributed computing concepts.
- Strong experience in data modeling, schema handling, and performance tuning.
- Excellent debugging, analytical, and problem-solving skills.
- Ability to work effectively in a hybrid and collaborative environment.
