Responsibilities :
Key Responsibilities AI Assurance Architecture
Architect platforms and frameworks for AI assurance, evaluation, and benchmarking
Design systems for LLM, agent, and RAG evaluation across functional, non functional, and risk dimensions
Define architectural patterns for Responsible AI, bias detection, explainability, and safety validation
Build reusable assurance components supporting Business Assurance, Risk Assurance, and Reliability Security, Reliability & Governance
Architect AI testing and validation for security, privacy, prompt injection, and adversarial robustness
Integrate red teaming, threat simulation, and chaos style validation for AI systems
Define governance mechanisms for model usage, auditability, traceability, and compliance
Ensure AI systems meet enterprise standards for resilience, fault tolerance, and observability Platform & Engineering Enablement
Design AI assurance platforms supporting automated test execution, reporting, and insights
Enable integration with CI/CD pipelines to enforce AI quality gates
Collaborate with QE engineering teams to embed AI assurance into the SDLC
Mentor teams on AI risk identification and mitigation from an engineering perspective Core Platforms, Frameworks & Tooling
LLM and AI evaluation frameworks (PromptFoo, DeepEval, custom LLM evaluation harnesses)
Prompt, RAG, and agent validation tooling (prompt testing frameworks, retrieval accuracy validators, agent workflow evaluators)
Responsible AI and model risk tooling (Fairlearn, SHAP, Explainable AI libraries, toxicity and bias scanners)
Security and adversarial testing tools for AI systems (PyRIT, Garak)
AI red teaming and threat simulation frameworks (automated red team scripts, adversarial test suites for LLMs and agents)
AI assurance automation and QE frameworks (Galileo)
Observability for AI behavior and drift (Langfuse, Arize, Evidently, custom telemetry dashboards) Client Orientation & Leadership
Partner with product and engineering teams to identify AI Assurance opportunities and shape roadmaps
Support client workshops, RFPs, and solution presentations
Mentor engineers on AI/ML/Gen AI best practices and emerging technologies
Translate complex AI concepts into business-friendly narratives
Technical and Professional Requirements:
Must Have Qualifications
13+ years of experience in software engineering with 3+ years in AI with strong architecture ownership
Hands on expertise in AI/ML systems, LLM evaluation, and assurance frameworks
Experience with AI red teaming, model risk management, or AI audit tooling
Strong understanding of Responsible AI, AI risks, and governance principles
Experience with security testing, adversarial testing, and reliability engineering
Proficiency in Python, automation frameworks, and cloud platforms Good to Have Skills
Knowledge of regulatory or compliance considerations for AI systems
Exposure to performance engineering, chaos engineering, or resilience testing for AI
Contributions to internal platforms, frameworks, or standards
