Data Quality (DQ) Observability
- Familiarity with cloud platforms and containerization technologies (Data Bricks).
- Proficiency in scripting and automation using languages such as Python,
- Excellent problem-solving skills with the ability to analyse complex issues and provide efficient solutions.
- Good communication skills and the ability to collaborate effectively across teams.
- Understanding of Agile and DevOps and GIT principles and their application in observability and monitoring contexts.
- Solid experience with instrumentation practices, including metrics, logging, and distributed tracing.
- E2E implementation. Right from requirement gathering and DQ design engaging with the Data Asset owners, development, and reporting.
- Development DQ rules with the help of Databricks
- Presentation of DQ results to the Data Asset Owners and other partners and work with them to build an action plan to improve the DQ results.
- Putting in place specific Data Improvement Plans
- Scale up the DQ framework to all the data assets and more.
- Ensure systems run efficiently, monitoring system performance and handle support in demanding environment.
- Handle the configuration of existing toolset for data quality, KPI system and Reporting.
Proficiency in implementing and maintaining observability tools is a plus
Qualifications And Experience Required
- Degree level education, preferably with overall 6-8 years of experience in SQL, Data warehousing and hands on Data Science
- Must be proficient and have experience in Data Science related technologies.
- Strong Python/ PySpark and SQL development skills
- Experience in Azure Databricks
- Agile development methodology using DevOps and GIT repos.
- Experience with Big Data tools and Cloud environment, especially Azure is a big plus.
- Expertise in converting Functional requirements to technical solutions and present in the business context.
- Experience in working with Hub and Spoke model data lakes, preferably with Diageo context is a big plus.
- Keeping pace with IT trends, innovation, and new developments in technology space
- Good Communication Skills required
