Associate Manager, D&T, Data Engineering

Job Title:Associate Manager, D&T, Data Engineering

Location- Hyderabad

Job Content:

Design and maintain scalable batch and real-time data pipelines on Databricks using PySpark, Structured Streaming, DBT, and Delta Live Tables. Manage cloud infrastructure with Terraform on AWS and deploy workflows using Databricks Asset Bundles with CI/CD best practices. Optimize Spark perfor mance and costs while implementing data governance through Unity Catalog. Collaborate with Data Scientists to produce ML models using MLflow and ensure data quality through automated testing and robust deployment pipelines.

Your keyresponsibilities:

• Design, build, and maintain scalable batch and real-time data pipelines using PySpark and Structured Streaming on Databricks, including migration of legacy ETL processes to Delta Live Tables (DLT).
• Develop and manage robust transformation layers using DBT to deliver clean, tested, and well-documented data models.
• Implement Infrastructure as Code (IaC) using Terraform to provision and manage AWS resources (S3, IAM, Glue) and Databricks workspaces.
• Deploy and manage data workflows using Databricks Asset Bundles (DAB) with standardized CI/CD practices via GitHub Actions or similar tools.
• Optimize Spark workloads for performance and cost efficiency, leveraging Photon Engine, Auto Loader, and best-practice tuning techniques.
• Enforce data governance and quality standards using Unity Catalog (row-level security) and support MLOps initiatives through MLflow-based model tracking and production deployment.
  
  

You bring:

• 10+ years of experience in Data Engineering, with strong expertise in the Databricks platform.
• Deep hands-on experience with Unity Catalog, Delta Live Tables (DLT), and Databricks Asset Bundles (DAB).
• Strong proficiency in AWS services (S3, Lambda, Kinesis) and Infrastructure as Code using Terraform.
• Expert-level programming skills in Python (PySpark) and SQL for scalable data processing.
• Proven experience in data transformation and modeling using DBT.
• Solid understanding of data modeling concepts, including Star Schema, Dimensional Modeling, and Data Vault, with the ability to contribute to architectural design decisions.
  
  

We bring:

• A team of diverse employees who aren't afraid to think outside of the box.
• A truly global and collaborative team that cares about the experience of our employees.
• The encouragement you need to develop and achieve personal growth.
• A role that is crucial on projects and allows you to build your brand.
• A caring and supportive environment where you're empowered to grow and share your ideas.
  
  

The application processes:

Interested in this position Please apply on-line by uploading your resume in English via our career portal. For further information, please contactNisha Tiwari, Talent Acquisition ([Confidential Information])

Inclusion, belonging and equal opportunity statement:

At dsm-firmenich, we believe being a force for good starts with the way we treat each other. When people feel supported, included, and free to be themselves, they do their best work - and that's exactly the kind of culture we're building. A place where opportunity is truly equal, authenticity is celebrated, and everyone has the chance to grow, contribute, and feel they belong.We're proud to be an equal opportunity employer, and we're serious about making our hiring process as fair and inclusive as possible. From inclusive language and diverse interview panels to thoughtful sourcing, we're committed to reflecting the world we serve.We welcome candidates from all backgrounds - no matter your gender, ethnicity, sexual orientation, or anything else that makes you, you.And if you have a disability or need any support through the application process, we're here to help - just let us know what you need, and we'll do everything we can to make it work.

Agency statement:

We're managing this search directly at dsm-firmenich. If you're applying as an individual, we'd love to hear from you. We're not accepting agency submissions or proposals involving fees or commissions for this role.