Data Engineer
Location: Chennai, Hyderabad, or Bangalore
Years of Experience: 6-9 years
Job Summary: We are seeking an experienced Data Engineer with 6-9 years of overall experience, including at least 3 years working with AWS Airflow. In this role, you will be instrumental in building and maintaining data pipelines that drive business insights and decision-making. You will collaborate closely with stakeholders to ensure data reliability, scalability, and availability for analytics and applications, helping the organization unlock the full value of its data assets.
Responsibilities
- Design, build, and maintain robust data pipelines to support analytics and business operations.
- Collaborate with business and technical teams to understand data requirements and deliver effective solutions.
- Ensure data quality, consistency, and availability across multiple systems and environments.
- Monitor, troubleshoot, and optimize data workflows to maintain high performance and reliability.
- Support data governance initiatives by implementing standards for data management and security.
Mandatory Skills
- AWS Airflow (workflow orchestration, DAG design, scheduling, monitoring) - Advanced (3+ years hands-on)
- AWS Cloud Services (S3, Lambda, Glue, Redshift, IAM) - Advanced
- Data Engineering Fundamentals (ETL, data pipelines, batch/stream processing) - Expert
- SQL (query optimization, joins, transformations) - Advanced
- Python (data manipulation, scripting, automation) - Advanced
Preferred Skills
- Data Governance & Security: Ensures compliance and protects sensitive data across pipelines.
- Communication & Collaboration: Needed to work effectively with distributed teams and business stakeholders.
- Performance Optimization: Important for scaling pipelines and reducing costs in cloud environments.
- Documentation & Knowledge Sharing: Critical for reproducibility and onboarding new team members.
- Problem Solving & Troubleshooting: Required to quickly resolve workflow issues and maintain reliability.
Qualifications: Bachelor's degree in Computer Science, Engineering, or a related field.
