Description
Were Hiring : Senior Databricks Engineer / Azure Data Engineer / Lead Data engineer.
Location : Baner, Pune (Onsite Only).
Work Mode : Full-time | Office-based.
Availability : Immediate Joiners / Notice Period 30 Summary :
We are looking for a Senior Databricks Engineer / Azure Data Engineer who will be responsible for designing and implementing high-performance, scalable, and secure data pipelines using Databricks, Spark, Delta Lake, and cloud platforms (Azure).
Youll collaborate with data architects, scientists, and business stakeholders to transform raw data into powerful insights that drive business impact.
Key Responsibilities
- Design, build, and maintain scalable data pipelines using Databricks, PySpark, and Spark SQL.
- Work with Databricks components such as Workspace, Jobs, Delta Live Tables (DLT), Repos, and Unity Catalog.
- Implement Delta Lake and Lakehouse (Medallion) architectures for optimized data storage and performance.
- Integrate CI/CD pipelines using Azure DevOps, GitHub Actions, or equivalent tools for Databricks deployments.
- Ensure data quality, lineage, and governance using Unity Catalog or Azure Purview.
- Use Airflow, Azure Data Factory, or Databricks Workflows for scheduling and orchestration.
- Apply strong SQL, data modeling, and ETL/ELT concepts in solution design.
- Collaborate with cross-functional teams to translate business requirements into technical deliverables.
Required Skills & Qualifications
- 7+ years of experience in Data Engineering, with a strong focus on Databricks.
- Expertise in PySpark, Spark SQL, and optionally Scala.
- Hands-on experience with Databricks components : Workspace, Jobs, DLT, Repos, Unity Catalog.
- Strong understanding of Delta Lake and Lakehouse architectures.
- Cloud proficiency in Azure (preferred), AWS, or GCP.
- Experience in CI/CD pipeline automation for Databricks deployments.
- Familiarity with orchestration tools (ADF, Airflow, Databricks Workflows).
- Strong grasp of data governance, metadata management, and lineage.
- Excellent communication and stakeholder management skills.
(ref:hirist.tech)
