Role : QA/Tester (AI)
About the Role:
The QAAI Tester will be responsible for validating the quality, accuracy, and reliability of AI-driven applications. The role involves designing and executing test plans, evaluating system outputs, identifying defects, and ensuring that the AI models and application functionalities meet defined standards. The tester will work closely with development and product teams to verify performance, usability, and consistency across various scenarios, ensuring a robust and dependable end-user experience
Experience
Key Responsibilities:
- Test Planning & Design: Collaborate with business analysts and development teams to understand requirements, and design comprehensive test plans and test cases that cover functional, non-functional, and performance testing for AI-powered systems.
- Test Execution: Execute test cases, report defects, and verify fixes in various environments (e.g., development, QA, staging) to ensure software quality before production release. Maintain defect tracking using industry-standard tools, document test results, and generate quality metrics reports.
- Automation & Scripting: Develop and maintain automated test scripts using testing frameworks and tools to enhance testing efficiency and coverage.
- Documentation: Maintain detailed documentation of test cases, test results, and defect reports, ensuring traceability and clarity for stakeholders.
- Collaboration: Work closely with cross-functional teams, including developers and product managers, to communicate testing results, discuss issues, and ensure alignment on quality standards.
- Continuous Improvement: Participate in retrospectives and contribute to process improvements within the testing lifecycle, promoting a culture of quality within the organization.
- Conduct hallucination testing to ensure AI system responses are factually accurate and free from fabricated information
- Test multimodal input processing: text queries, image handling, audio processing, and cross-modal consistency validation
- Validate output format compliance, communication standards adherence, and terminology usage accuracy
- Perform semantic search accuracy testing: query-to-document matching, embedding quality, similarity threshold optimization
- Execute load testing, stress testing, and performance benchmarking to ensure systems meet stringent latency and throughput requirements
- Validate compliance with organizational security standards, data protection policies, and regulatory requirements. Conduct comprehensive security testing, authentication/authorization, input validation, API security, data encryption verification
- Collaborate with development teams for bug reproduction, root cause analysis, and comprehensive fix verification
Job Description (JD)
- QA/Testing experience with at least 2 years in AI/ML, chatbot, or NLP system testing
- Bachelor's degree in computer science, Information Technology, or related technical field
- Strong understanding of AI/ML testing methodologies: model evaluation, data quality testing, bias detection
- Hands-on experience testing conversational AI systems, RAG pipelines, or LLM-based applications
- Expertise in functional, integration, system, and regression testing for complex software systems
- The Tester will be responsible for ensuring the quality and reliability of applications through comprehensive testing processes.
- This role involves designing test plans, executing test cases, and documenting results while collaborating with development and product teams to address issues and ensure compliance with quality standards. Proficiency in test case design, test planning, defect life cycle management, and quality assurance processes
- Ideal candidates should have a solid understanding of software testing methodologies, experience with test automation tools, test automation frameworks and strong analytical skills.
- Excellent attention to detail and communication skills are essential for effectively conveying testing outcomes and collaborating with stakeholders.
- Strong analytical and problem-solving skills for identifying edge cases, boundary conditions, and quality issues
- Excellent documentation skills for creating test plans, test reports, traceability matrices, and quality dashboards
- Experience coordinating UAT with stakeholders and managing iterative feedback loops
- Familiarity with multimodal AI testing: image classification validation, OCR accuracy, audio transcription quality
Educational Qualification
- Any Graduate (preferably B. E / B. Tech / MCA or equivalent degree in IT/CS) with good academic record from a recognized and reputed university/institution
Desirable Skills and Certificates
- AI-Specific Testing: Prompt testing, response quality evaluation, hallucination detection, factual accuracy validation
- Test Automation: Selenium, Playwright, Cypress for web UI automation, pytest for backend testing, Cucumber
- API Testing: Postman, REST Assured, pytest-requests, GraphQL testing tools
- Performance Testing: JMeter, Locust, Gatling, K6 for load testing, stress testing, and scalability testing
- Python Programming: Test automation scripts, data validation, metric analysis, custom testing tools
- Database Testing: SQL queries, data validation, NoSQL database query validation, data integrity checks
- LLM Evaluation Metrics: BLEU, ROUGE scores, semantic similarity metrics, retrieval metrics (MRR, nDCG, precision@k)
- Vector Database Testing: Query validation, similarity search accuracy, indexing verification, performance testing
- Test Management Tools: JIRA, TestRail, Zephyr, Azure DevOps Test Plans, test case management
- CI/CD Integration: Jenkins, GitHub Actions, GitLab CI for automated test execution and reporting
- Monitoring & Analytics Tools: Grafana, Kibana for test metrics visualization and trend analysis
- Browser DevTools: Chrome DevTools, Network analysis, Console debugging, performance profiling
- Mock Services & Stubs: WireMock, Mockito for service simulation and isolated component testing
Last date to apply : 26.01.26
