Responsibilities:
- Test AI-driven products including LLMs, RAG systems, AI assistants, chatbots, and ML prediction features.
- Validate AI outputs for accuracy, relevance, consistency, hallucination, bias, safety, and business correctness.
- Test prompt behavior, system instructions, guardrails, model responses, and AI workflow quality.
- Work with local LLM tools such as Ollama, LM Studio, LocalAI, or similar platforms.
- Create AI test datasets, prompt regression suites, golden test sets, and evaluation rubrics.
- Perform functional, regression, integration, API, UI, database, and end-to-end testing.
- Build and maintain automation scripts using Selenium, Playwright, Cypress, REST Assured, PyTest, or similar tools.
- Use Java, Python, or JavaScript to support automation and technical validation.
- Validate APIs, databases, logs, and backend workflows.
- Use AI-powered QA tools to improve test case creation, automation support, defect analysis, and QA productivity.
- Identify risks, requirement gaps, and quality issues early.
- Work independently with minimum guidance and take ownership of assigned deliverables.
- Collaborate with developers, business analysts, product owners, and QA team members.
Required Competencies/ Skills
- Bachelor's degree in computer science, Information Technology, or a related field
- Minimum of 7+ years of hands-on experience in manual and automation testing.
- 3+ years of experience testing AI, ML, LLM, chatbot, automation, or data-driven products is preferred.
- Good understanding of AI/ML/LLM concepts.
- Knowledge of RAG systems, embeddings, vector search, prompts, tokens, and hallucination risks.
- Hands-on experience with API testing and SQL.
- Automation experience using Selenium, Playwright, Cypress, REST Assured, PyTest, or similar tools.
- Programming knowledge in Java, Python, or JavaScript.
- Ability to analyze logs, troubleshoot issues, and validate backend data.
- Strong written and verbal communication skills.
- Ability to work independently and responsibly.
- Experience with vector databases.
- Demonstrate experience leading QA teams and mentoring senior and junior engineers
