Search by job, company or skills

I

RPA-GenAI Trust and Assurance Consultant

Save
  • Posted 17 hours ago
  • Be among the first 10 applicants
Early Applicant

Job Description

  • BTech/BE or equivalent technical degree.
  • 3–9 years of experience in testing/quality engineering, with hands-on exposure to Generative AI or LLM testing initiatives.
  • Working knowledge of AI testing concepts, including test design, evaluation metrics, and result interpretation for LLM outputs.
  • Experience contributing to structured documentation such as test plans, test evidence, and defect/risk reporting.
  • Define and execute testing strategies for Generative AI/LLM systems, covering functional, non-functional, and safety-focused validation.
  • Design evaluation frameworks and test suites using tools such as DeepEval to measure quality, robustness, and consistency of model outputs.
  • Plan and conduct red teaming exercises to identify vulnerabilities (e.g., prompt injection, jailbreaks, harmful content, data leakage) and recommend mitigations.
  • Establish Responsible AI assurance checks aligned to fairness, transparency, privacy, and safety expectations.
  • Create clear test plans, evidence, dashboards, and reports that communicate risks, findings, and remediation priorities to stakeholders.
  • Collaborate with engineering and product teams to integrate AI testing into delivery pipelines and improve release readiness criteria.
  • Support incident triage and root-cause analysis for model behavior issues, regressions, and evaluation drift across versions.
  • Proven experience executing red teaming for LLM applications and translating findings into actionable controls and mitigations.
  • Practical experience implementing Responsible AI practices and assurance workflows across the AI lifecycle.
  • Strong understanding of LLM failure modes (hallucinations, toxicity, bias, prompt sensitivity) and methods to test and reduce them.
  • Experience building automated evaluation pipelines and regression suites for LLM systems using DeepEval or similar frameworks.
  • Consulting experience: stakeholder management, requirement discovery, and delivering clear outcomes under ambiguity.

More Info

Job Type:
Industry:
Employment Type:

About Company

Job ID: 148918697

Similar Jobs

Bengaluru, India

Skills:

red teaming Test DesignTest PlansResponsible AIautomated evaluation pipelinesLLM testingevaluation metricsAI testing conceptsdefect risk reportingDeepEvalregression suitestest evidenceGenerative AI