AI Governance Engineer (LLM & Model Evaluation)

Responsible AI & Governance Lead Chennai
Build trust in intelligence design, evaluate, and govern AI systems responsibly.

Location: Chennai, India
Experience: 47 years total, with 1.53 years in GenAI, AI Governance, MLOps, or Model Evaluation.
Professionals who bring structure, ethics, and rigor to how AI systems are built, tested, and deployed.

Role Overview

As a Responsible AI & Governance Lead, you'll define and implement frameworks that ensure our GenAI and LLM systems are accurate, safe, reliable, and compliant. You'll bridge the worlds of technology, compliance, and ethics embedding governance directly into development workflows. This role is ideal for those passionate about AI accountability, model evaluation, and responsible innovation.

What makes you right for this role

• Model Evaluation & Assurance Design and operationalize frameworks to evaluate LLMs for accuracy, bias, safety, and reliability.

• Continuous Evaluation Pipelines Build automated offline and online eval systems for model benchmarking, regression testing, and drift detection.

• AI Guardrails & Safety Define and enforce policies for model output filtering, privacy protection, and content moderation.

• Governance Integration Embed evaluation and governance checks into CI/CD pipelines for AI product releases.

• Documentation & Auditability Maintain AI risk registers, model cards, approval workflows, and audit trails for transparency.

• Failure Analysis & Red Teaming Lead structured testing to identify hallucinations, data leaks, and unsafe completions before deployment.

• Regulatory Alignment Collaborate with compliance, legal, and data protection teams to meet standards like GDPR, HIPAA, RBI, SEBI, or ISO 27001.

The person you're

• Proficient in Python, with hands-on experience using evaluation and safety frameworks like DeepEval, OpenAI Evals, Traceloop, or Guardrails AI.

• Strong understanding of LLM pipelines RAG, fine-tuning, and agentic systems and how to integrate evaluation mechanisms within them.

• Skilled in designing both quantitative and qualitative metrics (toxicity, factuality, faithfulness, latency, token cost).

• Familiar with MLOps and observability stacks such as OpenTelemetry, Prometheus, Grafana, or Evidently AI.

• Knowledgeable about data governance, privacy, and security encryption, access control, anonymization, and retention policies.

• Experienced in regulated industries (finance, healthcare, public sector) emphasizing auditability and compliance-by-design.

• Exposure to ethical AI frameworks such as NIST AI RMF, OECD, or ISO 42001.

The mindset you bring

• Ethical Technologist You care deeply about the societal and operational impact of AI.

• Systems Thinker You see how evaluation, safety, and compliance interconnect.

• Detail-Oriented You thrive on accuracy, consistency, and traceability.

• Collaborative Partner You engage with data, product, and legal teams to ensure responsible innovation.

• Continuous Learner You stay ahead of evolving AI governance standards and frameworks.

Crayon Data Tech Stack

Python | DeepEval | OpenAI Evals | Traceloop | Guardrails AI | Evidently AI | OpenTelemetry | Prometheus | Grafana | RAG | LangChain | Azure | AWS | GCP