Senior Test Engineer - AI/ML

While technology is the heart of our business, a global and diverse culture is the heart of our success. We love our people and we take pride in catering them to a culture built on transparency, diversity, integrity, learning and growth.

If working in an environment that encourages you to innovate and excel, not just in professional but personal life, interests you- you would enjoy your career with Quantiphi!

Role:Senior Test Engineer - AI/ML

Experience: 3-6 years

Location: Mumbai/Bangalore (Hybrid)

Key Responsibilities:

• Design, develop, and maintain UI and API automation test suites using modern automation frameworks.

• Independently own end-to-end testing activities including test planning, automation development, execution, defect management, and reporting.

• Validate AI/ML applications, Generative AI solutions, LLM-powered applications, and intelligent workflows.

• Design and execute functional, integration, regression, and end-to-end test automation for traditional and AI-enabled applications.

• Develop and maintain reusable automation frameworks, libraries, and utilities.

• Automate API testing using REST Assured (Java) or Python Requests.

• Develop UI automation using Playwright.

• Validate backend data using SQL and perform database verification.

• Develop automated validations for AI model outputs, prompt responses, RAG pipelines, and chatbot conversations.

• Validate AI responses for correctness, consistency, relevance, grounding, safety, hallucinations, and bias.

• Collaborate with ML Engineers, Software Engineers, and Product Owners to define AI quality validation strategies.

• Integrate automated test suites into CI/CD pipelines and implement quality gates.

• Leverage GenAI tools to accelerate test case generation, automation development, debugging, documentation, and productivity.

• Contribute to AI testing frameworks, accelerators, and Quality Engineering best practices.

• Mentor junior engineers and promote automation-first and AI-first Quality Engineering practices.

Must have skills:

• Strong understanding of Software Testing Life Cycle (STLC), SDLC, Agile, and Scrum methodologies.

• Hands-on experience in designing test strategies, test scenarios, test cases, and test execution.

• Strong expertise in UI automation using Playwright and API automation using REST Assured (Java) or Python Requests.

• Hands-on experience with Postman for API testing and validation.

• Strong SQL skills with hands-on experience in database testing and backend data validation.

• Experience integrating automation suites with CI/CD pipelines using Jenkins, GitHub Actions, or similar.

• Proficiency with Git and version control best practices.

• Hands-on experience testing AI/ML or Generative AI applications.

• Understanding of Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), chatbots, copilots, or AI agents.

• Experience validating AI responses for accuracy, relevance, consistency, hallucinations, toxicity, bias, and safety.

• Experience designing test scenarios for prompt engineering and AI workflow validation.

• Understanding of AI evaluation techniques and automated output validation.

• Familiarity with AI evaluation frameworks or libraries (e.g., DeepEval, Ragas, LangSmith, Promptfoo, MLflow, or similar).

• Experience using GenAI tools such as ChatGPT, GitHub Copilot, Kiro, Claude, or similar to accelerate test design, automation development, debugging, test data generation, and documentation.

• Strong debugging, root cause analysis, and analytical problem-solving skills.

• Ability to independently own end-to-end testing activities with minimal supervision.

• Excellent communication, collaboration, and stakeholder management skills.

• Ability to mentor junior engineers and promote automation and Quality Engineering best practices.

• Strong attention to detail with a quality-first and automation-first mindset.

Nice to have skills

• Knowledge of AWS services such as Amazon Bedrock, SageMaker, EC2, Lambda, API Gateway, S3, RDS, DynamoDB, CloudWatch, ECS/EKS, and IAM.

• Experience testing cloud-native applications deployed on AWS.

• Experience in Data Warehouse, ETL, or Data Pipeline testing.

• Experience with performance testing tools such as JMeter or k6.

• Exposure to Docker, Kubernetes, and containerized application testing.

• Relevant certifications in Playwright, AWS, Quality Engineering, or Agile are an added advantage.