Description and Requirements
Responsibilities
- Build and lead the AI Quality Engineering organization at India, covering functional testing, scenario testing, automation, and regression.
- Define and execute a comprehensive AI testing strategy across Windows, Android, cloud, and cross-device experiences.
- Architect the AI automation framework for UI, system interactions, agent flows, and multimodal input/output validation.
- Own model and agent evaluation pipelines, including accuracy, latency, grounding, hallucination detection, safety checks, and guardrail tests.
- Build tooling for validating local versus cloud inference paths, RAG reliability, perception models, and agent orchestration flows.
- Partner with SRE & Delivery to define quality gates, release criteria, and gating policies for the release train.
- Create a unified scenario library across features like Learning Zone, Notion integration, cross-device continuity, Local Agent, and more.
- Develop a risk-based testing strategy to support parallel prototyping while maintaining high confidence in integration and release.
- Work closely with Platform, Core Intelligence, Experience, and Cloud teams to identify quality gaps early and enforce quality standards.
- Lead the cultural shift toward testability by design and raise the quality bar across all engineering teams.
Essential Skills
- 10+ years of experience in quality engineering, test automation, or AI/ML validation, with at least 3 years leading teams.
- Strong background in building automation frameworks for complex distributed or cross-device products.
- Hands-on experience testing AI models, RAG/LLM systems, perception models, or agentic behaviors.
- Deep experience with UI automation (Windows, Android), API testing, performance validation, and scenario-based testing.
- Experience working with telemetry, logs, and debugging across large-scale systems.
Additional Skills & Qualifications
- Ability to work across geographies and manage cross-functional alignment with platform, intelligence, cloud, and experience teams.
- Proven success driving quality culture in fast-paced engineering organizations.
- 25% global travel capability.
