Design, build, and optimize scalable data pipelines and ETL workflows on Databricks. Architect and implement data lakehouse solutions integrating Databricks with Azure Data Lake, Azure Synapse, and other Azure services.
Ensure security, compliance, and governance of data platforms, including RBAC, Unity Catalog, and data lineage.
Implement data quality, monitoring, and performance optimization strategies across large-scale datasets.
Define and enforce best practices for Databricks development, deployment, and CI/CD automation.
Troubleshoot and resolve issues related to Spark jobs, cluster performance, and data pipelines.
Mentor junior engineers and contribute to technical leadership and architectural decisions.
