Data Engineer

Position: Data Engineer – Azure Data Factory & Synapse

Location: Bangalore

Experience: 8+ Years

About the Role

Latinum is hiring for a client seeking an experienced Data Engineer with strong hands-on expertise in Azure Data Factory (ADF), Azure Synapse Analytics, SQL, and Power BI.

The ideal candidate will have extensive experience designing, developing, and maintaining scalable ETL/ELT pipelines across multiple enterprise data sources. Strong expertise in ADF pipeline orchestration, Azure DevOps-based CI/CD, and Power BI semantic modelling is essential.

The role involves building data solutions from the ground up, with a strong focus on clean architecture, scalability, maintainability, and performance. Familiarity with AI-assisted development tools such as ChatGPT and Claude will be valuable, as the team is progressively adopting AI-augmented and agent-assisted engineering workflows.

Key Responsibilities

• Design, develop, and maintain end-to-end ETL/ELT pipelines using Azure Data Factory and Azure Synapse Analytics.
• Manage pipeline orchestration, scheduling, dependency management, error handling, monitoring, and performance optimisation.
• Integrate data from multiple enterprise sources, including Oracle, Salesforce, and other business applications.
• Develop and manage ADF pipelines using Git-based version control and Azure DevOps.
• Implement CI/CD practices, including repository management, branching strategies, pull requests, and code reviews.
• Onboard new source systems by defining extraction strategies, incremental and daily loading mechanisms, and connectivity standards.
• Build and maintain Power BI semantic models, datasets, reports, and dashboards.
• Develop and support data transformations using Databricks wherever required.
• Create and maintain technical documentation, including data mapping documents, pipeline specifications, and data model documentation.
• Collaborate with business analysts and stakeholders to understand requirements, identify gaps, and translate business needs into technical solutions.
• Participate in testing, deployment, troubleshooting, and post-deployment validation.
• Leverage AI coding assistants such as ChatGPT and Claude for development support, documentation, and initial code reviews.
• Review and validate AI-assisted or agent-generated code and pull requests, ensuring adherence to engineering standards and production quality.

Required Skills & Experience

• 8+ years of hands-on Data Engineering experience.
• Strong, hands-on expertise in Azure Data Factory (ADF) and Azure Synapse Analytics.
• Excellent SQL skills and experience working with complex data transformations.
• Strong understanding of ADF pipeline architecture, orchestration, scheduling, monitoring, error handling, and performance optimisation.
• Practical experience with Azure DevOps and Git-based CI/CD, including repositories, branching, pull requests, and code management.
• Hands-on experience with Power BI, particularly semantic modelling, datasets, and DAX.
• Experience integrating Salesforce data through REST/Bulk APIs, SOQL, or connector-based approaches.
• Experience integrating data across multiple enterprise source systems, including Oracle and other business applications.
• Strong understanding of ETL/ELT patterns, data warehousing concepts, and enterprise data architecture.
• Ability to independently develop scalable, maintainable data pipelines and solutions from scratch.
• Experience using AI-assisted coding tools as part of a professional software development workflow.
• Strong technical documentation, analytical, troubleshooting, and problem-solving skills.

Good to Have

• Working knowledge of Azure Databricks, including PySpark or Scala.
• Experience integrating additional enterprise platforms such as SAP, ServiceNow, Workday, or similar systems.
• Exposure to AI-assisted development tools, coding agents, and automated code review workflows.
• Experience working in engineering teams transitioning towards automated and agent-driven pipeline development.