Description
- Lead the design, development, and implementation of scalable data pipelines and ELT processes using Databricks, DLT, dbt, Airflow, and other tools.
- Collaborate with stakeholders to understand data requirements and deliver high-quality data solutions.
- Optimize and maintain existing data pipelines to ensure data quality, reliability, and performance.
- Develop and enforce data engineering best practices, including coding standards, testing, and documentation.
- Mentor junior data engineers, providing technical leadership and fostering a culture of continuous learning and improvement.
- Monitor and troubleshoot data pipeline issues, ensuring timely resolution and minimal disruption to business operations.
- Stay up to date with the latest industry trends and technologies, and proactively recommend improvements to our data engineering practices.
Qualifications
- Systems (MIS), Data Science or related field.
- 15 years of experience in data engineering and/or architecture, with a focus on big data technologies.
- Extensive production experience with Databricks, Apache Spark, and other related technologies.
- Familiarity with orchestration and ELT tools like Airflow, dbt, etc.
- Expert SQL knowledge.
- Proficiency in programming languages such as Python, Scala, or Java.
- Strong understanding of data warehousing concepts.
- Experience with cloud platforms such as Azure, AWS, Google Cloud.
- Excellent problem-solving skills and the ability to work in a fast-paced, collaborative environment.
- Strong communication and leadership skills, with the ability to effectively mentor and guide
- Experience with machine learning and data science workflows
- Knowledge of data governance and security best practices
- Certification in Databricks, Azure, Google Cloud or related technologies.
This job posting will remain open a minimum of 72 hours and on an ongoing basis until filled.
Job Engineering
Primary Location India-Karnataka-Bengaluru
Schedule: Full-time
Travel: No
Req ID: 252684
Job Hire Type Experienced Not Applicable #BMI N/A
