Role: Sr. AI QA Engineer
Location: Noida / Gurugram / Pune / Mumbai
Employment Type: Full-Time
About the Role
We are looking for a highly skilled AI QA Engineer to join our Quality Engineering team. This role combines traditional automation testing expertise with modern AI application validation. The ideal candidate will have hands-on experience testing GenAI applications, LLM-powered systems, chatbots, RAG applications, and AI-driven workflows, along with strong automation experience using Playwright.
You will work closely with Product, Engineering, Data Science, and AI teams to ensure the quality, reliability, and accuracy of next-generation AI-powered products.
Key Responsibilities
- Design, develop, and maintain test automation frameworks using Playwright and Selenium.
- Perform end-to-end testing of web applications and AI-powered user experiences.
- Validate chatbot conversations, AI-generated responses, and conversational workflows.
- Execute testing for GenAI, LLM, and RAG (Retrieval-Augmented Generation) applications.
- Perform prompt testing, grounding validation, and hallucination detection.
- Validate AI agent workflows, multi-step reasoning, and tool integrations.
- Execute functional, regression, integration, API, database, and UAT testing.
- Perform SQL-based backend validations and data quality checks.
- Collaborate with cross-functional teams to define and implement quality strategies.
- Participate in Agile ceremonies and contribute to continuous quality improvement.
Must-Have Qualifications
- Bachelor's degree in Engineering (B.E./B.Tech) in Computer Science, Information Technology, Electronics, or related fields.
- Strong hands-on experience with Playwright (Mandatory).
- Experience with Selenium Automation.
- Experience testing GenAI, LLM, Chatbot, or AI-powered applications.
- Hands-on experience with RAG Testing.
- Strong understanding of:
- Prompt Testing
- Hallucination Testing
- Response Validation
- Grounding Validation
- Experience with API Testing using Postman or REST APIs.
- Strong SQL skills and experience with databases such as:
- MySQL
- PostgreSQL
- MongoDB
- DynamoDB
- Experience with Python or Java.
- Familiarity with:
- Git
- Jira
- CI/CD Pipelines
- Agile Methodologies
Good-to-Have
- Experience with Agentic AI Testing
- Experience with MCP (Model Context Protocol)
- Exposure to LangSmith, Ragas, Promptfoo, DeepEval
- Experience with vector databases:
- Pinecone
- ChromaDB
- Weaviate
- FAISS
- AWS or GCP experience
- Exposure to Amazon Bedrock or Azure OpenAI
Preferred Project Experience
Candidates with experience in any of the following will be highly preferred:
GenAI & Chatbot Testing
- Conversational AI testing
- Prompt validation
- LLM response evaluation
- Hallucination detection
RAG Application Testing
- Retrieval quality validation
- Embedding and chunking validation
- Knowledge grounding verification
- Citation accuracy testing
Agentic AI Testing
- Multi-agent workflows
- Autonomous decision-making systems
- AI workflow orchestration
- Tool and API integrations
Skills
Playwright, Selenium, Generative AI, LLM Testing, RAG Testing, AI Testing, Chatbot Testing, Prompt Testing, Agentic AI, Python, Java, API Testing, Postman, SQL, AWS, CI/CD, Jira, Test Automation, Quality Assurance
Why Join Us
Join a team building innovative AI-powered products where you'll play a critical role in ensuring reliability, accuracy, and exceptional user experiences through advanced AI quality engineering practices.
