Search by job, company or skills

Data Engineer

Data Engineer

DNEG
3-5 Years
Not Disclosed

This job is no longer accepting applications

Job Description

As a Data Engineer, you’ll architect and maintain the pipelines that power our products and services. You’ll work at the intersection of ML, media processing, and infrastructure; owning the data tooling and automation layer that enables scalable, high-quality training and inference. If you’re a developer who loves solving tough problems and building efficient systems, we want you on our team.

Key Responsibilities

  • Design and maintain scalable pipelines for ingesting, processing, and validating datasets with main focus visual and voice data.
  • Work with other teams to identify workflow optimisation potential, design and develop automation tools, using AI-driven tools and custom model integrations and scripts.
  • Write and maintain tests for pipeline reliability.
  • Build and maintain observability tooling in collaboration with other engineers to track data pipeline health and system performance.
  • Collaborate with data scientists, operators, and product teams to deliver data solutions.
  • Debug and resolve complex data issues to ensure system performance.
  • Optimise storage, retrieval, and caching strategies for large media assets across environments.
  • Deploy scalable data infrastructure using cloud platforms as well as on-premise and containerization.
  • Deepen your knowledge of machine learning workflows to support AI projects.
  • Stay current with industry trends and integrate modern tools into our stack.

Must Haves

  • 3+ years in data engineering or related backend/infrastructure role.
  • Strong programming skills in Python or similar languages.
  • Experience with software development lifecycle (SDLC) and CI/CD pipelines.
  • Proven experience building and testing data pipelines in production.
  • Proficiency in Linux.
  • Solid SQL knowledge.
  • Experience with Docker or other containerisation technologies.
  • Proactive approach to solving complex technical challenges.
  • Passion for system optimisation and continuous learning.
  • Ability to adapt solutions for multimedia data workflows.

Nice to Have

  • Experience with Kubernetes (k8s).
  • Knowledge of machine learning or AI concepts.
  • Familiarity with ETL tools or big data frameworks.
  • Familiarity with cloud platforms (e.g., AWS, GCP, Azure).

About You

  • Innovative
  • Like challenges
  • Adaptable
  • Calm under pressure
  • Strong communication abilities

More Info

Job Type:
Industry:
Employment Type:

Key Skills

AI-driven tools

big data frameworks

About Company

Similar Jobs

5-15 yrs
₹ 5.5 - 18 LPA
Bengaluru
Skills:
Scala, Python, Spark
7-9 yrs
Bengaluru, India
Skills:
Pyspark, AWS Glue, Data Modeling, Emr, Avro, Autosys, Sql, Cloudwatch, Terraform, Etl Tools, Databricks, Python, AWS, Airflow, Parquet, AWS Lake Formation, Iceberg
5-10 yrs
Bengaluru, India
Skills:
Power Bi, Pyspark, Agile Scrum, Tableau, Databases, Git, Data Warehousing, Informatica Powercenter, Python, Airflow, Data Lake concepts, Advanced SQL and performance tuning, IICS, dbt, CI/CD
7-11 yrs
Bengaluru, India
Skills:
Kms, Pyspark, Iam, Sql, Python, AWS Lake Formation
7-9 yrs
Bengaluru, India
Skills:
Java, Python, AWS Analytics, data modeling and database design, data warehousing concepts and architecture, data governance and data quality frameworks