Platform Solution Architect Azure & Databricks

Role Overview:

We are looking for an experienced Platform Solution Architect with strong expertise in Azure & Databricks to design and implement enterprise-scale data platforms.

The role will focus on platform architecture, reusable frameworks, data quality, observability, governance, and engineering standards.

Key Responsibilities

Platform Architecture & Solution Design:

• Define Lakehouse and Medallion architecture standards.
• Design Delta architecture, partitioning, file layout and schema evolution.
• Build observability into the platform architecture.
• Create reference architectures, HLD/LLD and design decision records.
• Lead solution and architecture design reviews.

Reusable Framework Engineering

• Design metadata-driven frameworks for source onboarding.
• Build reusable ELT/transformation frameworks.
• Implement standard error handling, restartability, idempotency and audit logging.
• Design framework-level observability including telemetry, audit counts, watermark and SLA tracking.

Data Quality & Observability

• Design Data Quality frameworks covering validation, quarantine, reconciliation and quality scoring.
• Architect end-to-end data observability for freshness, completeness, volume anomalies, schema drift, lineage and job health.
• Use Databricks System Tables, job/cluster telemetry, Delta History and Azure Monitor/Log Analytics.
• Build platform health dashboards for pipeline reliability, data quality and operational KPIs.

Engineering Standards

• Define CI/CD and environment promotion for Databricks assets using Azure DevOps and Git.
• Implement automated testing and deployment observability.
• Ensure release traceability, environment drift detection and post-deployment validation.

Required Skills

• 12+ years of experience in Data Engineering/Data Platform roles.
• 5+ years in Platform or Solution Architecture.
• Enterprise-scale Azure Databricks Architecture.
• Databricks workspace design, compute strategy and job orchestration.
• Unity Catalog design, governance, fine-grained access control and lineage.
• Metadata-driven ingestion and reusable ELT framework design.
• Data Quality framework design.
• Data and pipeline observability.
• Databricks System Tables, job telemetry and Delta History.
• Azure Monitor / Log Analytics.
• HLD/LLD and architecture diagramming.
• Technical leadership and mentoring across Data Engineering, Security, DevOps and Cloud teams.

Qualifications

• Must have owned at least one enterprise-scale Azure Databricks platform from architecture through production operation.
• Proven experience designing reusable frameworks, Data Quality and Observability capabilities adopted across multiple teams.
• Experience working across Data Engineering, Security, DevOps and Cloud functions in an enterprise environment.