Job Description
Role: QA Engineer Generative AI / Responsible AI Testing
Experience: 47 Years
Location: Open to any Orion Innovation Office in India ( Chennai/Pune/ Mumbai/Kochi/Coimbatore/Hyderabad)
Employment Type: FTE
Role Overview
We are seeking a detail-oriented QA Engineer with experience in Generative AI and Responsible AI testing to ensure the quality, safety, and reliability of AI-powered applications, including LLM-driven chatbots and intelligent systems. The role will focus on functional, adversarial, and ethical validation of GenAI solutions while supporting automation and performance testing initiatives.
Key Responsibilities
GenAI & Responsible AI Testing
- Design and execute prompt-based test cases covering functionality, bias, safety, and edge scenarios.
- Craft adversarial prompts to stress-test LLM behavior and identify vulnerabilities.
- Validate hallucination resistance, output consistency, and prompt reliability.
- Evaluate AI systems for bias, fairness, transparency, and ethical compliance.
- Test content moderation and safety filters for GenAI applications.
- Perform end-to-end QA for GenAI chatbot conversational flows.
Functional & System Testing
- Design test scenarios, test cases, and regression suites.
- Conduct System Integration Testing (SIT), Smoke Testing, Regression Testing, and Bug Testing.
- Execute manual and automated test cycles across releases.
- Maintain defect logs, triage defects, and track resolution.
Automation & Performance Testing
- Develop and maintain automation scripts using Python-based frameworks.
- Build automated tests using Pytest / Selenium / Robot Framework.
- Support load and performance testing to assess system scalability.
BI & Data Validation Testing
- Validate BI dashboards, reports, and visualizations.
- Ensure data accuracy, reporting reliability, and performance across tools like Tableau and Power BI.
- Support data storytelling and analytics validation.
Agile & DevOps Collaboration
- Participate in Sprint Planning, Backlog Grooming, and Daily Stand-ups.
- Collaborate with Developers, Product Owners, and Analysts to streamline QA processes.
- Support build deployments across QA, UAT, and Production environments.
- Provide feature demos and stakeholder updates.
Required Skills & Competencies
Testing Expertise
- Software Testing Life Cycle (STLC) & SDLC
- Test case design & execution
- Defect management & triaging
- Regression & integration testing
GenAI / AI Testing
- LLM testing & validation
- Prompt engineering for testing
- Responsible AI principles
- Bias & fairness evaluation
Automation & Tools
- Python
- Pytest
- Selenium / Robot Framework
- LangChain (preferred)
BI & Data Tools
- Tableau
- Power BI
- Advanced Excel
Project & QA Tools
- Jira
- Confluence
- Azure DevOps
Qualifications
- Bachelor's Degree in Computer Science, Engineering, or related field.
- ISTQB CTFL certification preferred.
Nice to Have
- Knowledge of Machine Learning / Deep Learning concepts.
- Experience with GenAI governance frameworks.
- Exposure to statistical analysis and data science fundamentals.
Soft Skills
- Strong analytical & problem-solving skills
- Excellent communication & stakeholder management
- Attention to detail
- Ability to work in Agile teams
