Role Title: Databricks Architect / Lead – Healthcare Data Platform
Experience: 15–20 Years
Location: Remote / Client Engagement Model
Domain: Healthcare & Life Sciences
Role Overview
We are seeking an experienced
Databricks Architect / Lead with strong expertise in building
modern healthcare data platforms on the Databricks Lakehouse architecture. The role will focus on designing scalable, secure, and compliant data solutions that support healthcare analytics, clinical insights, regulatory reporting, and AI/ML initiatives.
The candidate will work closely with
data engineering teams, healthcare domain experts, and enterprise architects to enable advanced analytics and data interoperability while ensuring compliance with healthcare data standards and privacy regulations.
Key Responsibilities
Data Platform Architecture
- Design and implement Databricks Lakehouse architecture for healthcare data platforms.
- Build scalable pipelines using Apache Spark, Databricks Delta Lake, and Databricks Workflows.
- Architect ingestion frameworks for structured and semi-structured healthcare data from EMR/EHR systems, claims systems, and clinical repositories.
- Implement data governance, lineage, and security frameworks within the Databricks ecosystem.
Healthcare Data Integration
- Integrate healthcare data sources including:
- Electronic Health Records (EHR)
- Claims and billing systems
- Clinical data repositories
- Healthcare IoT and medical devices
- Enable interoperability using healthcare standards such as:
- FHIR
- HL7
- DICOM (for imaging datasets where applicable)
Data Engineering & Analytics
- Develop high-performance ETL/ELT pipelines using Databricks.
- Optimize large-scale healthcare datasets using Delta Lake optimization techniques.
- Enable analytics for:
- Population health
- Clinical outcomes
- Claims analytics
- Healthcare operations and financial analytics
- Support AI/ML initiatives using Databricks MLflow and machine learning pipelines.
Security, Compliance & Governance
- Implement secure data architectures aligned with healthcare compliance standards such as:
- HIPAA
- HITRUST
- PHI data protection frameworks
- Implement data masking, encryption, and role-based access control.
- Enable auditing, monitoring, and regulatory reporting.
Collaboration & Leadership
- Work with clinical data teams, business analysts, and product owners to translate healthcare use cases into scalable data solutions.
- Provide technical leadership to data engineering and analytics teams.
- Define data platform standards, architecture guidelines, and best practices.
- Support client engagements and contribute to solution architecture discussions.
Required Skills & Experience
Databricks & Data Engineering
- Strong hands-on experience with Databricks Lakehouse Platform
- Expertise in Apache Spark (PySpark/Scala).
- Experience with Delta Lake architecture and optimization.
- Building data pipelines using Databricks workflows or orchestration tools.
Cloud Platforms
Experience In One Or More Cloud Platforms
- Microsoft Azure (preferred)
- AWS
- Google Cloud Platform
Data Architecture
- Experience with data lake, data warehouse, and lakehouse architectures
- Knowledge of data modeling (Medallion Architecture, dimensional models).
- Experience working with large-scale healthcare datasets.
Healthcare Domain Knowledge
- Understanding of healthcare data models and regulatory frameworks.
- Experience with FHIR, HL7, ICD codes, CPT codes, and claims data models.
- Knowledge of clinical analytics and healthcare reporting.
DevOps & Automation
- Experience with CI/CD for data pipelines.
- Infrastructure as Code using Terraform / ARM / CloudFormation.
- Monitoring and performance tuning of data pipelines.
Preferred Qualifications
- Databricks certifications (Data Engineer / Architect).
- Experience implementing Real World Evidence (RWE) analytics platforms.
- Exposure to GenAI or AI-driven healthcare analytics.
- Experience working with payer, provider, or life sciences organizations.
Education
- Bachelor's or Master's degree in Computer Science, Data Engineering, ata lakehouse platform.
- Enable secure access to PHI-compliant healthcare data.
- Support advanced analytics, AI/ML, and clinical insights.
- Drive data modernization initiatives within healthcare organizations.
