We are looking for a Senior Azure Data Engineer (Databricks) who can independently design, develop, and optimize cloud-based data solutions. The ideal candidate must be strong in Azure Databricks, Data Lakehouse architecture, ETL/ELT development, and data warehouse modelling. This role requires hands-on expertise in building and enhancing pipelines, migrating enterprise data (SAP/SQL Server) to cloud, and implementing scalable solutions using Azure services. This is a development-heavy role (initially 100% development, later 60% development) with ownership of end-to-end implementation projects.
Location: Gurugram
Exp: 7-10 years
Shift timing: 2-11pm
Roles & Responsibilities
- Work on Azure Databricks pipelinesenhance, modify, and optimize existing pipelines and develop new ones when required.
- Handle data migration from SAP/SQL Server and enterprise databases into Azure cloud environments (Lakehouse, Delta, Synapse).
- Configure, manage, and optimize Databricks clusters, compute, workspaces, and resources for performance and cost efficiency.
- Build scalable ETL/ELT processes using PySpark, SQL, Delta Lake, and medallion architecture across BronzeSilverGold layers.
- Work extensively on Azure Data Factory, orchestration pipelines, triggers, monitoring, and integration with Databricks.
- Design and maintain data warehouse & dimensional models (Facts, Dimensions, SCD, Star/Snowflake schema).
- Ensure strong data quality, validation, governance, and securityincluding Unity Catalog, RBAC, and Azure platform controls.
- Develop solutions independently, involving end-to-end development, not just supportinitially 100% development-focused.
Skill Set Required
- Azure Databricks (Notebooks, Clusters, Delta Lake, Optimization, Resource Configuration)
- Azure Data Lake / Lakehouse Architecture
- Azure Data Factory (Pipelines, Orchestration, Integration, Monitoring)
- Azure SQL / SQL Server
- PySpark, SQL for transformation and performance tuning
- Data Warehouse Fundamentals
o Star & Snowflake schema
o Fact & Dimension modelling
o SCD, Data Marts
- Azure Cloud Services (Key Vault, Storage, Synapse basics, RBAC, security)
- ETL/ELT development (end-to-end implementation)
