Job Title: Data Quality Analyst
Job Summary:
We are looking for a detail-oriented Data Quality Analyst to monitor, investigate, and improve data quality across our Databricks platform. The role involves identifying anomalies, analysing data issues, and ensuring data accuracy and reliability using tools like Databricks Data Quality Monitoring, Unity Catalog lineage, and Lakeflow observability. You will work closely with Data Governance, Data Engineering, and Analytics teams to maintain high-quality, business-ready data.
Key Responsibilities:
- Monitor Databricks data quality alerts and identify anomalies or inconsistencies.
- Investigate data issues using Unity Catalog lineage and Lakeflow observability.
- Perform data profiling and validation using SQL and Python.
- Assist in implementing data quality checks using Delta Lake constraints or DLT expectations.
- Analyse issues such as schema drift, data volume changes, and late-arriving data.
- Collaborate with Data Engineers to integrate automated quality checks in pipelines.
- Document investigations, processes, and remediation steps.
- Support data governance initiatives and data quality reporting.
Required Skills:
- Hands-on experience with Databricks, SQL, and Python.
- Strong data analysis and troubleshooting skills.
- Understanding of data quality concepts (accuracy, completeness, consistency, timeliness).
- Experience examining datasets to identify anomalies.
- Strong communication and documentation skills.
Preferred Skills:
- Experience with Unity Catalog lineage and Delta Live Tables (DLT).
- Knowledge of data validation frameworks, CI/CD, and Git.
- Cloud or Databricks certifications are a plus.
