Search by job, company or skills

DP World

Group Lead - Data Quality Engineer

DP World
Bengaluru, India
8-10 Years
Save
new job description bg glownew job description bg glow
  • Posted 2 hours ago
  • Be among the first 10 applicants
Early Applicant

Job Description

Job Description

KEY ACCOUNTABILITIES

  • Data Quality Policy & Framework Implementation
  • Define and operationalize enterprise Data Quality policies, procedures, and standards.
  • Establish standardized data quality dimensions and certification frameworks.
  • Implement scalable validation frameworks across ingestion, transformation, and serving layers.
  • Embed quality-by-design principles into data product lifecycle.
  • Data Observability Platform Development
  • Design and implement end-to-end data observability capabilities including:
    • Data freshness and SLA monitoring
    • Volume and distribution anomaly detection
    • Schema drift and pipeline health monitoring
    • Data lineage validation and reliability tracking
    • Develop automated alerting and incident detection mechanisms.
  • Custom Data Applications (DataApps) Development
  • Build custom Data Quality and Observability applications using:
    • Databricks native capabilities
    • Streamlit / Databricks Apps
    • Python-based backend services
  • Develop user interfaces enabling:
    • Data quality rule configuration
    • Dataset certification workflows
    • Quality score visualization
    • Issue tracking and remediation workflows
    • Enable self-service quality monitoring for engineering and analytics teams.
  • Azure & Databricks Platform Integration
  • Implement data quality checks within Azure-based data pipelines and Databricks workflows.
  • Integrate monitoring with:
    • ADLS Gen2
    • Databricks Lakehouse architecture
    • Batch and streaming pipelines
    • Develop reusable frameworks leveraging Spark and Delta Lake.
    • Optimize performance and scalability of quality validation workloads.
  • Automation & Engineering Excellence
  • Integrate DQ checks into CI/CD and deployment pipelines.
  • Develop metadata-driven quality monitoring solutions.
  • Implement automated remediation and self-healing workflows where applicable.
  • Ensure auditability, traceability, and governance compliance.
  • Metrics, Reporting & Adoption
  • Define enterprise Data Quality KPIs and reliability SLAs.
  • Build dashboards tracking platform-wide data trust scores.
  • Drive adoption of standardized DQ practices across engineering teams.
  • Support audit and compliance reporting initiatives.
  • Data Quality Score
  • Leadership & Collaboration
  • Act as technical lead for Data Quality and Observability engineering.
  • Mentor engineers on best practices for data reliability.
  • Collaborate with Data Engineering, Governance, and Platform Architecture teams.
  • Contribute to long-term evolution of the enterprise data platform.
Qualifications, Experience And Skills

Education

  • Bachelor's or master's degree in computer science, Data Engineering, Information Systems, or related field.

Experience

  • 8+ years of experience in Data Quality engineering roles within Data Platforms/Data Engineering teams.
  • Proven experience building custom applications on Databricks or data platforms.
  • Experience designing enterprise Data Quality or Data Observability solutions.
  • Hands-on experience developing internal data tools or platform applications.

Technical Skills (Required)

  • Cloud & Data Platform

Strong expertise in:

  • Microsoft Azure
  • Databricks Lakehouse platform
  • ADLS Gen2
  • Distributed data processing using Spark
  • Application & DataApp Development
  • Experience building DataApps using:
    • Streamlit
    • Databricks Apps or notebook-based applications
    • Python backend development
    • Experience designing UI-driven data engineering tools or internal platforms.
  • Data Quality & Observability
    • Experience implementing data validation frameworks.
    • Strong SQL and Python programming skills.
    • Knowledge of anomaly detection, monitoring, and data reliability concepts.
  • Engineering & Integration
    • CI/CD integration for data pipelines.
    • REST API integrations and automation workflows.
    • Metadata-driven architectures and lineage concepts.
Core Competencies

  • Platform-first engineering mindset.
  • Strong problem-solving and analytical thinking.
  • Ability to translate governance requirements into scalable technical solutions.
  • Strong stakeholder collaboration and communication skills.
  • Ownership mindset with ability to lead initiatives end-to-end.

Preferred

  • Experience with Great Expectations, Deequ, Soda, or similar frameworks.
  • Experience with streaming data validation.
  • Exposure to AI-driven data observability or anomaly detection.
  • Experience building enterprise internal developer platforms.

More Info

Job Type:
Permanent Job
Industry:
Other
Function:
Data Engineering
Employment Type:
Full time

About Company

DP World

Job ID: 147505721

Jobs by Skill - IT
Jobs by Skill - Non IT
International Jobs
Last Updated: 16-05-2026 00:17:57 AM
Homejobs in Bengaluru / BangaloreGroup Lead - Data Quality Engineer

Similar Jobs

Group Lead - Data Quality Engineer

DP World
D
Bengaluru, India

Skills:

Sql, Microsoft Azure, Databricks Apps, Distributed data processing using Spark, CI CD integration for data pipelines, Python backend development, Databricks Lakehouse platform, ADLS Gen2, Streamlit, REST API integrations, Metadata-driven architectures, Data Quality Observability