Location: Bangalore / Mumbai
Role Overview
Seeking a QA Engineer with AI/ML testing experience to ensure the quality, reliability, and performance of AI-powered applications. The role involves validating AI models, LLM solutions, and data pipelines while identifying issues such as hallucinations, bias, model drift, and data quality concerns.
Key Responsibilities
- Test AI/ML models, LLM applications, RAG systems, chatbots, and data pipelines.
- Develop automated validation, regression, and performance testing frameworks.
- Evaluate model outputs for accuracy, consistency, relevance, bias, and hallucinations.
- Monitor production models for drift and performance degradation.
- Validate data quality and conduct fairness, bias, and explainability assessments.
Must-Have Skills
- Bachelor's/Master's degree in Computer Science, Engineering, or related field.
- 3 6 years of QA experience, including 1 2 years in AI/ML testing.
- Strong Python skills for automation and data analysis.
- Experience with RAGAS, DeepEval, Promptfoo, LangSmith, PyTest, Selenium, and Postman.
- Good understanding of the ML lifecycle, data quality validation, and pipeline testing.
- Strong analytical, communication, documentation, and stakeholder management skills.
Good-to-Have Skills
- Prompt Engineering and LLM Red Teaming.
- MLOps tools: MLflow, SageMaker, Vertex AI.
- Vector Databases, Responsible AI, AI Safety, and Fairness Frameworks.
- A/B Testing, Shadow Deployments, CI/CD, and DevOps practices.
