Job Title: QA Engineer
Location: Bengaluru, Karnataka (On-Site)
Experience: 2-8 Years
Role Overview
We are looking for a QA Engineer to lead validation of AI use cases and solution approaches, ensuring that proposed solutions are robust, reliable, and aligned with problem requirements before productionization.
Key Responsibilities
- Define evaluation frameworks for AI/LLM-based use cases
- Design and execute advanced test scenarios for model behavior and edge cases
- Validate solution approaches during prototyping and experimentation phases
- Work closely with stakeholders to ensure solution correctness and feasibility
- Establish metrics for evaluating model performance and output quality
- Guide teams on best practices for AI validation and testing
- Mentor junior QA engineers in analytical and evaluation approaches
Required Skills
- Strong understanding of AI/ML systems and LLM behavior
- Experience in testing complex, non-deterministic systems
- Ability to define evaluation frameworks and validation strategies
- Strong analytical and problem structuring skills
Good to Have
- Experience with LLM evaluation tools and techniques
- Exposure to prompt engineering and model tuning
- Familiarity with Python and experimentation workflows
What We're Looking For
- Strong judgment and critical thinking
- Ability to validate is this the right solution vs just does it
