Databricks Lead – Delivery & Engineering1
Lead hands-on Databricks delivery, ensuring adherence to CoE standards and best practices.
Key Responsibilities
- Lead Databricks engineering teams
- Build and optimize Spark & Delta Lake pipelines
- Implement Databricks Jobs, Workflows, CI/CD
- Optimize performance and DBU usage
- Contribute to Databricks accelerators and standards
- Design and implement end-to-end data platforms using Databricks Lakehouse
- Lead development of:
- Data ingestion (batch & streaming)
- Delta Lake data modeling (Bronze/Silver/Gold)
- Scalable ETL/ELT pipelines
- Optimize Spark workloads for performance and cost
- Ensure reliability, scalability, and observability of data pipelines
Experience
- 10–12 years in data engineering
- 3–5 years hands-on Databricks experience
Skills
- Deep expertise in Databricks, Apache Spark, Delta Lake
- Strong hands-on experience with PySpark / Spark SQL (core, SQL, streaming)
- Experience designing Lakehouse architectures
- Strong understanding of data modeling, data quality, and lineage
- Databricks platform features
- Cloud-native data engineering
- Performance tuning & troubleshooting
Certifications (Preferred)
- Databricks Data Engineer Associate and Professional
