Job Title: Senior Azure Data Engineer
Experience: 912 Years
Location: Noida
Job Description
We are looking for a
Senior Azure Data Engineer with extensive experience in building and optimizing large-scale data pipelines and architectures using the
Microsoft Azure ecosystem. The ideal candidate should have deep hands-on expertise in
Azure Data Factory (ADF),
Azure Databricks, and
Unity Catalog, along with a strong understanding of
data governance, security, and performance optimization.
The role involves designing scalable, reliable, and secure data platforms that enable data-driven decision-making across the enterprise.
Key Responsibilities
- Design and implement end-to-end data pipelines using Azure Data Factory (ADF) and Azure Databricks.
- Manage data ingestion, transformation, and orchestration across multiple sources and destinations.
- Implement Delta Lake architecture and manage data governance through Unity Catalog.
- Collaborate with data architects and business teams to define data integration strategies.
- Optimize pipelines for performance, scalability, and cost efficiency.
- Ensure data security, lineage, and cataloging across environments.
- Work closely with stakeholders to define and maintain data models and ETL/ELT frameworks.
- Implement and maintain CI/CD pipelines for data workflows using Azure DevOps or GitHub Actions.
- Drive best practices in data management, quality, and compliance.
- Mentor junior engineers and provide technical guidance on Azure data solutions.
Technical Skills
Core Azure Tools & Services:
- Azure Data Factory (ADF): Pipeline orchestration, triggers, integration runtimes
- Azure Databricks: PySpark, Delta Lake, Notebooks, job orchestration
- Unity Catalog: Data governance, access control, lineage, compliance
- Azure Synapse Analytics and Azure SQL Database
Programming & Scripting
- Strong proficiency in SQL and Python (PySpark)
- Experience with ETL/ELT design patterns
Data Storage & Architecture
- Azure Data Lake Storage (ADLS Gen2), Delta Lake, Parquet/CSV/JSON data formats
- Data modeling (star/snowflake schema)
DevOps & Automation
- CI/CD pipelines using Azure DevOps / GitHub
- Version control (Git)
- Infrastructure as Code (IaC): Terraform / ARM Templates (optional but preferred)
Other Preferred Skills
- Knowledge of Power BI / Data Visualization tools
- Experience with data quality frameworks
- Familiarity with Data Mesh / Modern Data Platform concepts
