Search by job, company or skills

Associate Delivery Manager - Data Engineer

Associate Delivery Manager - Data Engineer

BAJAJ FINSERV HEALTH
5-7 Years
Not Disclosed
Early Applicant
  • Posted 2 months ago
  • Be among the first 10 applicants

Job Description

Location Name: Pune Corporate Office - Mantri

Job Purpose

To effectively design, develop, and manage data solutions using ETL technologies such as Azure Databricks (ADB) , Azure Data Factory (ADF) and SQL along with leading 3 to 5 members of developers

Duties And Responsibilities

KEY RESPONSIBILITIES

  • Lead the end-to-end design, development, and delivery of scalable data engineering solutions using Azure Databricks, ADF, PySpark, SQL, and Delta Lake.
  • Convert business requirements into robust technical designs, architecture documents, data models, and implementation plans.
  • Own technical delivery of data integration, ETL, lakehouse, semantic layer, and AI/BI enablement initiatives.
  • Guide and mentor data engineers on coding standards, design best practices, performance optimization, and reusable framework development.
  • Review technical designs, code, pipelines, and deployment plans to ensure quality, scalability, maintainability, and compliance.
  • Drive architecture decisions for batch and near-real-time data pipelines across Bronze, Silver, and Gold layers.
  • Ensure data quality, reconciliation, anomaly detection, and timely resolution of production issues through effective RCA and permanent fixes.
  • Optimize data pipelines, Databricks jobs, SQL queries, and storage usage to improve performance and reduce cost.
  • Implement CI/CD practices, version control, automated deployments, and environment management across Dev, QA, and Production.
  • Collaborate with PMO, business stakeholders, BI teams, InfoSec, DevOps, and external partners for smooth project execution.
  • Establish SOPs, engineering standards, reusable components, monitoring frameworks, and documentation practices.
  • Track delivery progress, manage technical dependencies, prioritize work, and ensure timely closure of project milestones.
  • Support adoption of modern data platforms, semantic modeling, metrics layer design, and GenAI/BI capabilities.
  • Ensure compliance with data governance, security, access control, audit, and enterprise data management standards.
  • Act as the technical escalation point for critical issues, complex solutioning, and cross-team dependency resolution.

Key Decisions / Dimensions


KEY DECISIONS / DIMENSIONS

  • Define semantic layer design and metric definitions
  • Prioritize data vs AI optimization trade-offs
  • Handle production issues with RCA and long-term fixes
  • Drive architectural decisions for lakehouse + Data integration

Major Challenges


MAJOR CHALLENGES

  • Ensuring Data Delivery within TAT
  • Driving adoption of GenAI-based BI over traditional dashboards
  • Balancing performance, cost, and scalability
  • Managing dependencies across data engineering, AI, and business teams

Required Qualifications And Experience


REQUIRED SKILLS & EXPERIENCE

Must Have

  • Azure Databricks – PySpark, SQL, Delta Lake
  • Strong experience in Semantic Modeling & Metrics Layer design
  • Hands-on with Databricks workflows
  • Pyspark (Pandas, PySpark, FastAPI)
  • Azure Data Factory (ADF) for ETL pipelines
  • Strong SQL and data modeling skills

Good to Have

  • Cosmos DB / MongoDB (NoSQL concepts)
  • Azure Data Explorer (KQL)

DATA STACK (MANDATORY FOR SCREENING)

SNoData Platform / ConceptsAssociated Technologies

1Databricks LakehousePySpark, SQL, Delta Lake

2AI for BIDatabricks Genie, Genie Rooms, Instructions, Agents

4ETL & OrchestrationAzure Data Factory

5ProgrammingPyspark

6Cloud PlatformAzure (Preferred)

10DevOpsCI/CD Pipelines, Git



More Info

Job Type:
Industry:
Employment Type:

Key Skills

Metrics Layer design

Delta Lake

Databricks workflows