Job Description:

Step into a role where you help teams build Generative AI solutions that are not only powerful but also trustworthy safe and testable
As part of a collaborative consulting environment you ll work closely with product engineering and risk stakeholders to design practical assurance approaches for LLM powered systems balancing innovation with reliability
You ll contribute to real world AI validation efforts from defining test strategies and evaluation metrics to executing red teaming exercises and documenting outcomes that leadership can act on
If you enjoy combining structured testing discipline with the fast evolving world of GenAI this is a chance to shape how AI quality safety and responsibility are measured and improved while learning alongside motivated teams solving meaningful problems

Key Responsibilities:

Define and execute testing strategies for Generative AI LLM systems covering functional non functional and safety focused validation
Design evaluation frameworks and test suites using tools such as DeepEval to measure quality robustness and consistency of model outputs
Plan and conduct red teaming exercises to identify vulnerabilities e
g
prompt injection jailbreaks harmful content data leakage and recommend mitigations
Establish Responsible AI assurance checks aligned to fairness transparency privacy and safety expectations
Create clear test plans evidence dashboards and reports that communicate risks findings and remediation priorities to stakeholders
Collaborate with engineering and product teams to integrate AI testing into delivery pipelines and improve release readiness criteria
Support incident triage and root cause analysis for model behavior issues regressions and evaluation drift across versions

Technical Requirements:

BTech BE or equivalent technical degree
3 9 years of experience in testing quality engineering with hands on exposure to Generative AI or LLM testing initiatives
Working knowledge of AI testing concepts including test design evaluation metrics and result interpretation for LLM outputs
Experience contributing to structured documentation such as test plans test evidence and defect risk reporting

Additional Responsibilities:

Proven experience executing red teaming for LLM applications and translating findings into actionable controls and mitigations
Practical experience implementing Responsible AI practices and assurance workflows across the AI lifecycle
Strong understanding of LLM failure modes hallucinations toxicity bias prompt sensitivity and methods to test and reduce them
Experience building automated evaluation pipelines and regression suites for LLM systems using DeepEval or similar frameworks
Consulting experience stakeholder management requirement discovery and delivering clear outcomes under ambiguity

Preferred Skills:

Technology->Machine Learning->Generative AI,Technology->Generative AI->Prompt Engineering,Technology->Machine Learning->Responsible AI,Technology->Agentic AI->Agent Engineering,Technology->AI Engineering->LLMOps,Technology->Testing Technologyes->Test Automation Technology->Testing,Technology->Data Science->Machine Learning

RPA-GenAI Trust and Assurance Consultant