MODULE LEAD - Azure Databricks

• Design, develop, and maintain scalable data pipelines and ETL/ELT workflows on Azure.
• Implement and manage data solutions using Azure Data Factory, Azure Databricks, Azure Synapse Analytics, and Azure Data Lake Storage.
• Develop complex SQL queries, stored procedures, views, and optimize database performance.
• Design and implement data warehouse and data lake architectures.
• Integrate data from multiple structured and unstructured sources.
• Ensure data quality, governance, security, and compliance standards are met.
• Collaborate with business stakeholders, architects, analysts, and development teams to gather requirements and deliver solutions.
• Troubleshoot and resolve data processing, performance, and integration issues.
• Participate in code reviews, design discussions, and best practice implementation.
• Support CI/CD deployment processes and production releases.

Total Exp 7-10Y

• Azure Data Factory (ADF)
• Azure Databricks
• Azure Synapse Analytics
• Azure Data Lake Storage (ADLS Gen2)
• Azure SQL Database
• Azure Key Vault
• Azure DevOps

• Strong SQL expertise, including:
  • Complex query development
  • Performance tuning and optimization
  • Stored procedures, functions, triggers, and indexing strategies
• Strong experience in ETL/ELT design and development.
• Data Warehousing and Dimensional Modeling (Star/Snowflake Schemas).
• Hands-on experience with PySpark and Spark SQL.
• Experience handling large-scale data processing and optimization.
• Strong understanding of data governance, security, and best practices.