Role Overview
Looking for a Senior Data Engineer with strong experience in
Azure, Databricks, Spark, Python, and SQL to design, build, and maintain scalable data pipelines for large-scale retail data processing. The role involves data architecture, DataOps/DevOps implementation, and supporting analytics and machine learning initiatives.
Key Responsibilities
- Develop and maintain scalable ETL/ELT data pipelines using Python and SQL.
- Work with Azure, Databricks (Delta/Live Tables), Spark, Kafka, and Azure Event Hub for data processing.
- Design and implement robust data models, data architecture, and data quality frameworks.
- Build and manage CI/CD pipelines, automated testing, and Infrastructure as Code using GitHub Actions and Terraform.
- Collaborate with data scientists, product teams, and stakeholders.
- Mentor junior engineers and drive best practices in data engineering.
Required Skills
- 5+ years of Data Engineering experience.
- Strong expertise in Python and SQL.
- Hands-on experience with Azure, Databricks, and Apache Spark.
- Good knowledge of data warehousing, data modeling, ETL/ELT, and data quality.
- Experience with DataOps/DevOps, CI/CD, GitHub Actions, and Terraform.
- Familiarity with streaming technologies such as Kafka or Azure Event Hub.
- Strong communication and stakeholder management skills.
Ideal Candidate
A senior-level Data Engineer with expertise in
Azure Data Engineering, Databricks, Spark, Python, SQL, Data Architecture, and DevOps practices, capable of leading end-to-end data platform development and mentoring team members.
Skills: sql,azure,python,devops practices,spark,databricks
