Themandatory QA skill requirements areas follows:
#urgenthiring #immediatejoiners
Functional & Behavioral Testing:
- Test AI agents across multi-step workflows, goal completion, and decision-making paths
- Validate agent behavior under normal, edge, and failure scenarios
- Verify correct tool calling, API usage, and response handling
- Ensure consistency across retries, sessions, and memory states
Reasoning & Safety Validation:
- Evaluate agent reasoning quality (logic, coherence, hallucination risk)
- Test compliance with safety, policy, and ethical(HIPAA) constraints.
- Identify prompt injection, jailbreaks, or unintended autonomous actions
Automation & Evaluation:
- Design and maintain automated test suites for agent workflows.
- Create evaluation metrics for accuracy, latency, success rate, and failure recovery
- Use logs, traces, and conversation graphs to debug agent behavior
Data & Feedback Loops;
- Curate and maintain test datasets, test cases and scenarios
- Label and analyze failures to improve prompts, policies, and agent design
- Collaborate with ML, product, and engineering teams to close gaps.
- Experience testing AI/ML or LLM-based systems
- Understanding of prompting and prompt chaining,tool/function calling,state, memory, and context management
Good to Have skills:
- Experience with LLM frameworks (LangChain, AutoGen, CrewAI, Semantic Kernel, etc.)
- Familiarity with evaluation tools (LLM evals, red teaming, simulation testing)
- Knowledge of observability tools (traces, logs, replay tools)
- Exposure to security or safety testing
If interested please share your resume on [Confidential Information]
