The Lead Data Engineer is responsible for designing, building, and optimizing scalable data pipelines and backend workflows. This role focuses on delivering robust data solutions on AWS, enforcing engineering best practices, and providing technical leadership across data platforms and production systems.
Key Responsibilities
- Lead the design, development, and enhancement of ETL pipelines and backend data workflows using Python and SQL
- Design, implement, and optimize large-scale data processing solutions on AWS using services such as Glue, EMR, Lambda, and S3
- Lead the implementation and continuous improvement of CI/CD pipelines for automated build, testing, and deployment
- Translate business and technical requirements into scalable, secure data solutions aligned with approved architecture standards
- Write high-quality, maintainable, and well-tested code while setting strong engineering and design standards
- Diagnose and resolve complex technical issues, data quality concerns, and performance bottlenecks across environments
- Participate in and lead code reviews to ensure code quality, testing coverage, and documentation standards
- Mentor junior and mid-level engineers through technical guidance, feedback, and knowledge sharing
- Contribute to the evolution of data architecture, development processes, and engineering best practices
- Support and improve production data products and services across multiple environments
Qualifications
- Strong experience in data engineering or backend development roles
- Advanced proficiency in Python and SQL for data processing and transformation
- Hands-on experience with AWS data and compute services
- Solid understanding of CI/CD practices and automated deployment pipelines
- Experience leading technical initiatives and guiding engineering teams
