Your Role and Impact
We are seeking a highly skilled
Databricks Architect with deep expertise in
Databricks features including
Delta Lake, Unity Catalog, Databricks SQL, and strong experience in
Azure Data Factory (ADF). The ideal candidate will design and optimize large-scale data platforms, ensure governance and security compliance, and drive innovation in advanced analytics and machine learning workflows.
- 8+ years of extensive knowledge of Databricks features: Delta Lake, Unity Catalog, Databricks SQL, MLflow.
- Strong experience with Azure Data Factory for orchestration and integration.
- Hands-on expertise in performance optimization and writing complex SQL queries.
- Solid understanding of cloud architecture, security, and governance best practices.
- Excellent problem-solving and troubleshooting skills for large-scale data pipelines.
Preferred Experience: Experience in designing and managing large-scale data platforms and advanced analytics solutions.
Certifications
- Databricks Certified Data Engineer or Architect.
- Microsoft Azure certifications (e.g., Azure Data Engineer Associate).
Soft Skills
- Strong analytical and problem-solving abilities.
- Excellent communication and stakeholder management skills.
- Ability to lead and mentor technical teams.
Your Contribution
Architecture Assessment & Design
- Design and implement scalable, secure data architectures leveraging Databricks and Azure Data Factory.
- Develop frameworks for data ingestion, transformation, and analytics using Delta Lake, Unity Catalog, and Databricks SQL.
- Optimize Spark jobs, SQL queries, and pipelines for performance and cost efficiency.
- Collaborate with data engineers, analysts, and business stakeholders to deliver robust solutions.
- Implement governance, security, and compliance standards, including RBAC and data lineage.
- Ensure high availability and reliability through proactive monitoring and troubleshooting.
Strategic & Business Alignment
- Translate strategic business objectives into enterprise-wide data architecture solutions.
- Align data architecture with organizational priorities, ensuring data capabilities support analytics, reporting, and advanced use cases (AI/ML).
- Define and enforce architecture principles, standards, and governance frameworks.
Technical Leadership
- Guide cross-functional teams in adopting architectural patterns and best practices.
- Lead the selection of appropriate data platforms, tools, and frameworks.
- Champion the adoption of DataOps practices for faster, automated, and high-quality data delivery.
- Promote ETL automation using AI/ML-driven solutions to optimize data ingestion, transformation, and error detection.
Presales & Client Engagement
- Support presales and solution design activities, working alongside Sales, Product, and Customer Success teams.
- Participate in client discovery calls, architecture reviews, and solution presentations.
- Develop high-level architecture overviews and proof-of-concepts for client proposals.
