Job Title: Databricks & Microsoft Fabric Engineer (24 Years Experience)
Role Summary
We are seeking a Databricks & Microsoft Fabric Engineer with 24 years of hands-on experience in building scalable data platforms. This role focuses on implementing Lakehouse architectures using Databricks and Microsoft Fabric, while working across Azure Synapse, Azure Data Factory (ADF), Power BI, and CI/CD pipelines to deliver governed and analytics-ready data solutions.
Key Responsibilities
- Contribute to the design and implementation of Databricks + Microsoft Fabric Lakehouse solutions using Delta Lake and Medallion (BronzeSilverGold) architecture.
- Build and optimize PySpark-based pipelines in Databricks integrated with Fabric Lakehouse and OneLake.
- Develop and maintain ELT/ETL pipelines using Fabric Data Pipelines, Dataflow Gen2, Azure Data Factory, and Synapse.
- Support the integration of multiple source systems (finance, operations, HR, etc.) into curated and gold data layers.
- Create and maintain Power BI semantic models connected to Fabric Warehouse/Lakehouse.
- Implement data quality checks, monitoring, and logging using Python/PySpark.
- Assist in CI/CD implementation using Azure DevOps and Git for Fabric and Databricks artifacts.
- Apply data governance and security practices including RBAC, sensitivity labels, data masking, and Key Vault.
- Collaborate with business and analytics stakeholders to translate requirements into technical data solutions.
- Work closely with senior engineers and architects to enhance platform scalability and performance.
Required Skills & Experience
- 24 years of experience in Data Engineering / Analytics Engineering roles.
- Hands-on experience with Databricks (PySpark, Delta Lake, Jobs/Workflows).
- Working experience with Microsoft Fabric (Lakehouse, OneLake, Data Pipelines, Dataflow Gen2).
- Experience with Azure Data Factory, Synapse Analytics, and Power BI.
- Strong proficiency in Python, SQL (T-SQL), and PySpark.
- Practical exposure to Medallion / Lakehouse architectures.
- Experience working with Git and CI/CD pipelines (Azure DevOps preferred).
- Understanding of data governance, security, and best practices.
Preferred / Nice to Have
- Exposure to Azure ML / MLOps, AKS, or event-driven architectures.
- Familiarity with PowerApps / Power Automate.
- Experience working in enterprise or regulated environments.
Education
Bachelor's or Master's in Computer Science, Engineering, or a related field.
