Sr. Responsible AI Analyst

Responsibilities

1. Execute Responsible AI Evaluations: Conduct structured assessments on fairness, safety, robustness, explainability, and hallucination risks for models under development or deployment.

2. Design Evaluation Datasets and Metrics: Create curated, adversarial, and edge-case datasets and define quantitative metrics for evaluating fairness, toxicity, reliability, and ethical alignment.

3. Support Guardrail and Control Implementation: Contribute to implementing technical guardrails, safety filters, explainability modules, and automated checks within AI pipelines.

4. Analyze Ethical and Technical Risks: Review datasets, model outputs, and system behavior to identify fairness gaps, robustness issues, transparency deficiencies, and other Responsible AI risks. 5. Participate in Red Teaming and Stress Testing: Assist with scenario-based adversarial evaluations, prompt safety checks, robustness tests, and model vulnerability analysis.

6. Support Deployment of Responsible AI Workflows: Assist in implementing lifecycle governance workflows, templates, and processes—such as via IBM OpenPages or equivalent governance tooling.

7. Prepare Transparency and Governance Documentation: Develop model cards, system cards, evaluation reports, risk logs, and supporting documentation required for governance reviews and audit readiness.

8. Assist in Continuous Monitoring: Support creation of dashboards, metrics, and monitoring signals to track fairness drift, hallucination patterns, model instability, and safety deviations.

9. Collaborate Across Engineering and Governance Functions: Work closely with AI engineers, data scientists, product teams, legal, ISG, DPO, and governance bodies to ensure Responsible AI requirements are consistently applied.

10. Assist in Training and Knowledge Enablement: Help develop training content, guides, and resources to educate internal teams on Responsible AI evaluation methods, guardrails, and governance expectations.

Technical and Professional Requirements:

• Proficiency in Python and ML/DL frameworks (PyTorch, TensorFlow) for evaluation and experimentation.

• Understanding of fairness libraries (Fairlearn, AIF360) and ability to compute ethics related evaluation metrics.

• Familiarity with explainability tools (SHAP, LIME, Captum, Integrated Gradients).

• Exposure to red teaming concepts, prompt safety evaluation, and data integrity checks.

• Experience with MLOps basics including evaluation pipelines, experiment tracking, and CI workflows.

• Understanding of ML algorithms, generative models, and supervised/unsupervised learning techniques.

• Familiarity with NLP, vision, speech, and structured data domains.

• Knowledge of datasets, benchmark suites, and third party model ecosystems.

Preferred Skills:

Technology->AI-Generative AI->Generative AI - Basic->retrieval augmented generation (rag)

Technology->AI-Responsible AI->Responsible AI