Job description
We are looking for a highly motivated individual with strong analytical and technical skills for the position of AI Validation Engineer. You will play a critical role in evaluating the performance, accuracy, and fairness of our AI solutions, including the AI Assistant and its associated skills. You will help ensure that our AI systems meet the highest standards of quality and align with our responsible AI principles.
About the Role
In this opportunity as Manager, you will:
- Collaborate with data scientists, AI engineers, and product managers to understand AI system requirements and functionalities.
- Develop and execute comprehensive test plans to assess the performance, accuracy, robustness, and fairness of AI models and applications.
- Design and implement automated testing frameworks for efficient and continuous AI validation.
- Analyze AI model outputs, identify potential biases, errors, or limitations, and provide actionable feedback to the development team.
- Contribute to the development and implementation of responsible AI guidelines and best practices within the organization.
- Stay abreast of the latest advancements in AI validation techniques and technologies.
- Document testing procedures and results clearly and concisely.
- Collaborate with stakeholders to address and resolve identified AI validation issues.
About You
You're a fit for the role of Manager if your background includes:
- Team Leadership and Coordination: Lead and mentor a team of Responsible AI Partners and oversee the activities of the distributed Data and Model Stewards Network. Foster a collaborative environment focused on effective model governance.
- Policy and Framework Implementation: Drive the implementation and adoption of our Model Policies, Standards, and Ethics Framework across various business units and projects.
- Stakeholder Engagement: Act as a key point of contact and trusted advisor for scientists, product groups, content groups, and business stakeholders on all matters related to Responsible AI and model governance.
- Internal Consulting and Guidance: Provide expert guidance and support to project teams on embedding model stewardship principles and best practices throughout the model lifecycle.
- Risk Oversight and Mitigation: Ensure the effective identification, assessment, and mitigation of model risks within your portfolio of projects, working closely with data scientists and engineering teams.
- Performance Monitoring and Reporting: Establish and monitor key metrics to assess the effectiveness of our model governance framework and provide regular reports to leadership.
- Process Improvement: Continuously evaluate and enhance our model governance processes, identifying opportunities for increased efficiency, robustness, and alignment with evolving regulations and ethical considerations.
- Data Analysis and Storytelling: Leverage data analysis and visualization techniques to understand our content, processes, and the impact of our AI models. Narrate compelling stories based on data insights, often to non-technical audiences.
