While technology is the heart of our business, a global and diverse culture is the heart of our success. We love our people and we take pride in catering them to a culture built on transparency, diversity, integrity, learning and growth.
If working in an environment that encourages you to innovate and excel, not just in professional but personal life, interests you- you would enjoy your career with Quantiphi!
Role: Senior Test Engineer - AI/ML
Experience: 3-6 years
Location: Mumbai/Bangalore (Hybrid)
Key Responsibilities
- Design, develop, and maintain UI and API automation test suites using modern automation frameworks.
- Independently own end-to-end testing activities including test planning, automation development, execution, defect management, and reporting.
- Validate AI/ML applications, Generative AI solutions, LLM-powered applications, and intelligent workflows.
- Design and execute functional, integration, regression, and end-to-end test automation for traditional and AI-enabled applications.
- Develop and maintain reusable automation frameworks, libraries, and utilities.
- Automate API testing using REST Assured (Java) or Python Requests.
- Develop UI automation using Playwright.
- Validate backend data using SQL and perform database verification.
- Develop automated validations for AI model outputs, prompt responses, RAG pipelines, and chatbot conversations.
- Validate AI responses for correctness, consistency, relevance, grounding, safety, hallucinations, and bias.
- Collaborate with ML Engineers, Software Engineers, and Product Owners to define AI quality validation strategies.
- Integrate automated test suites into CI/CD pipelines and implement quality gates.
- Leverage GenAI tools to accelerate test case generation, automation development, debugging, documentation, and productivity.
- Contribute to AI testing frameworks, accelerators, and Quality Engineering best practices.
- Mentor junior engineers and promote automation-first and AI-first Quality Engineering practices.
Must Have Skills
- Strong understanding of Software Testing Life Cycle (STLC), SDLC, Agile, and Scrum methodologies.
- Hands-on experience in designing test strategies, test scenarios, test cases, and test execution.
- Strong expertise in UI automation using Playwright and API automation using REST Assured (Java) or Python Requests.
- Hands-on experience with Postman for API testing and validation.
- Strong SQL skills with hands-on experience in database testing and backend data validation.
- Experience integrating automation suites with CI/CD pipelines using Jenkins, GitHub Actions, or similar.
- Proficiency with Git and version control best practices.
- Hands-on experience testing AI/ML or Generative AI applications.
- Understanding of Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), chatbots, copilots, or AI agents.
- Experience validating AI responses for accuracy, relevance, consistency, hallucinations, toxicity, bias, and safety.
- Experience designing test scenarios for prompt engineering and AI workflow validation.
- Understanding of AI evaluation techniques and automated output validation.
- Familiarity with AI evaluation frameworks or libraries (e.g., DeepEval, Ragas, LangSmith, Promptfoo, MLflow, or similar).
- Experience using GenAI tools such as ChatGPT, GitHub Copilot, Kiro, Claude, or similar to accelerate test design, automation development, debugging, test data generation, and documentation.
- Strong debugging, root cause analysis, and analytical problem-solving skills.
- Ability to independently own end-to-end testing activities with minimal supervision.
- Excellent communication, collaboration, and stakeholder management skills.
- Ability to mentor junior engineers and promote automation and Quality Engineering best practices.
- Strong attention to detail with a quality-first and automation-first mindset.
Nice to have skills
- Knowledge of AWS services such as Amazon Bedrock, SageMaker, EC2, Lambda, API Gateway, S3, RDS, DynamoDB, CloudWatch, ECS/EKS, and IAM.
- Experience testing cloud-native applications deployed on AWS.
- Experience in Data Warehouse, ETL, or Data Pipeline testing.
- Experience with performance testing tools such as JMeter or k6.
- Exposure to Docker, Kubernetes, and containerized application testing.
- Relevant certifications in Playwright, AWS, Quality Engineering, or Agile are an added advantage.
If you like wild growth and working with happy, enthusiastic over-achievers, you'll enjoy your career with us!
