Description
Job Title : Databricks Lead.
Location : : 8-10 years of overall experience and 5+ years of hands-on leadership in data engineering projects.
Budget : Open to discuss.
Notice Period : Immediate joiner/ Serving notice with less than 60 days/Notice is less than 60 days.
About The Role
We are looking for a highly skilled Databricks Lead with 810 years of overall experience and 5+ years of hands-on leadership in data engineering projects.
The ideal candidate will lead teams in designing, building, and optimizing scalable data pipelines and Lakehouse solutions on Databricks.
Key Responsibilities
- Lead the development and delivery of end-to-end data pipelines using Databricks, Spark, Delta Lake, and PySpark.
- Drive implementation of Lakehouse/Medallion architectures (Bronze/Silver/Gold).
- Guide and mentor data engineers, ensure best coding and architectural practices.
- Optimize Spark jobs, Databricks clusters, and workflows for performance and cost.
- Implement CI/CD pipelines, version control, and DevOps practices for Databricks (Repos, Workflows, Terraform).
- Collaborate with architects, data scientists, and business teams to deliver scalable data solutions.
- Ensure data quality, governance, security, and compliance standards are followed (Unity Catalog preferred).
Required Skills & Experience
- 810 years in Data Engineering or Cloud Data Platforms.
- 5+ years leading data engineering teams or technical delivery.
- Strong hands-on experience with Databricks, Spark, PySpark, SQL, and Delta Lake.
- Experience with cloud platforms (AWS/Azure/GCP) and related data services.
- Solid understanding of data modeling, ingestion frameworks, orchestration, and performance tuning.
- Experience with CI/CD, Git, Terraform, Databricks Repos, and Workflows.
- Strong problem-solving, leadership, and communication skills.
Preferred Qualifications
- Databricks certifications (Associate or Professional).
- Experience with streaming technologies (Kafka, Kinesis, Event Hubs).
- Familiarity with MLflow and data science integration.
What We Offer
- Opportunity to lead cutting-edge data engineering initiatives.
- Collaborative, high-growth environment.
- Competitive compensation and professional development opportunities.
(ref:hirist.tech)
