Lead Software Engineer

Position Senior Azure Databricks Engineer Working Location PAN India NCR or Chennai preferred Experience 8 to 10 years Notice Period Preferably Immediate to 30 days Roles Responsibilities Collaborate with Data Platforms and Integration team in architecting and implementing scalable enterprise data platform solutions utilizing Azure Databricks Delta Lake Unity Catalog Azure Data Lake Storage Gen2 and related Azure services Design and develop enterprise data engineering frameworks supporting batch streaming ELT ETL and real time analytics workloads Administer and maintain Azure Databricks workspaces including cluster configuration job scheduling workspace administration Unity Catalog implementation secret management compute policies cluster policies and workspace governance Define and implement enterprise security architecture including identity integration with Microsoft Entra ID Azure AD Role Based Access Control RBAC Privileged Identity Management PIM data encryption private networking managed identities Key Vault integration and secure access patterns Establish enterprise data governance standards utilizing Unity Catalog including data lineage data quality metadata management catalog organization access policies and audit capabilities Optimize Databricks workloads through performance tuning Spark optimization cluster sizing workload isolation caching strategies partitioning Delta optimization Photon and cost optimization best practices Design enterprise monitoring logging operational dashboards alerting and platform health monitoring utilizing Azure Monitor Log Analytics Application Insights and Databricks system tables Develop reusable architecture patterns reference implementations development standards naming conventions coding standards operational procedures and platform documentation Support migration of legacy enterprise data warehouses ETL solutions and reporting platforms to Azure Databricks and modern Lakehouse architecture Provide technical support and troubleshooting for