Assistant Vice President Lead Databricks Architect
Shift Type: UK Shift
Work Model:5 Days Work From Office
Lead the design and delivery of large-scale data engineering solutions on Azure and Databricks Lakehouse. This role focuses on scalable data pipelines, Unity Catalog governance, ML/LLM enablement, and driving engineering excellence.
Key Responsibilities
- Architect, build, and optimize data pipelines using Databricks, ADF, PySpark, and Unity Catalog.
- Implement data quality, lineage, privacy, and security governance across data assets.
- Integrate data from multiple sources to support analytics, ML feature stores, and enterprise insights.
- Operationalize ML models using MLflow and enhance productivity using LLM/Copilot solutions.
- Lead cross-functional data initiatives end-to-end; mentor engineers and enforce coding & platform standards.
- Drive efficiency through CI/CD, DevOps, automation, and reusable framework components.
Required Qualifications
- 15+ years in Data Engineering with deep hands-on experience in:
- Databricks Lakehouse, ADF, PySpark, Azure Data Lake, Unity Catalog
- Strong background in ETL/ELT pipeline design, orchestration, and cloud data architecture.
- Experience with enterprise data governance, metadata, and lineage capabilities.
- Databricks Certification is mandatory (Data Engineer Associate/Professional).
- Azure Data Engineer / Cloud certifications preferred.
