Job Description
Required Skills & Qualifications:
This role requires deep awareness of quality challenges unique to LLM- and SLM-powered Agentic AI applications, especially in healthcare and education, where correctness, reliability, and compliance are essential.
- 7+ years in Quality Engineering/Automation, with 3 years in QA leadership roles.
- Proven experience transforming teams from manual QA to automation-first.
- Awareness of LLM/SLM quality challenges (latency unpredictability, token inefficiency, hallucinations, SME UAT cycles).
- Strong automation expertise (Playwright, PyTest, Cypress, JUnit, REST API testing).
- Familiarity with Agentic AI frameworks (LangChain, LangGraph, RAG pipelines, Vector DBs).
- Experience in healthcare or education applications with regulatory constraints.
- Solid background in CI/CD, DevOps, and cloud-native systems (Azure, Kubernetes, GitHub Actions).
Nice to Have (Big Plus)
- Experience with Playwright MCP (multi-context automation) for scaling automation.
- Hands-on with AI evaluation tools (Promptfoo, DeepEval, OpenAI Evals).
- Familiarity with AI observability & monitoring (Datadog).
- Background in AI security testing (prompt injection, adversarial robustness).
