Senior AI Governance Java Engineer

Role: Senior AI Governance Java Engineer

Experience: 4 to 7 years

Notice Period: Immediate to 10 days

Locations: Hyderabad/ Indore/ Ahmedabad.

About the Role:

We are seeking strong 4+ Yrs of Senior Java + AI Platform Engineers to work on strategic enterprise initiatives focused on building scalable AI Governance, Observability, and Compliance platforms.

Design and build the evaluator platform that powers an enterprise AI Governance offering a pluggable framework that runs evaluations against multiple model providers, classifies risk for EU AI Act readiness, and feeds outcomes into policy workflows. 

You will work as an embedded engineer in the AI Governance engineering team, contributing across design, implementation, testing and code review.

The ideal candidate will have strong expertise in Java backend engineering, distributed systems, AI integrations, telemetry pipelines, and cloud-native architectures. This role involves designing scalable enterprise-grade platforms capable of monitoring, governing, and securing AI interactions across large-scale enterprise environments.

Key Responsibilities:

● Design and build the evaluator service, leveraging native evaluator platforms where available and an endpoint-execution framework where they are not.

● Develop the evaluation orchestrator and the EU AI Act risk-classifier service (scheduling and execution of evaluation runs and workflows).

● Build pluggable cloud and model-provider integrations - AWS, Azure AI Foundry, Google Vertex AI, OpenAI and Anthropic.

● Build evaluation datasets and metric packs for toxicity, PII/privacy, accuracy & hallucination, fairness & bias, robustness, transparency, explainability, data quality and custom evaluators.

● Build scheduler capability and integrate with cloud deployment frameworks to deploy evaluators.

● Contribute to the self-serve UI enabling customers to generate datasets from their own ground truth, and to observability/dashboards for evaluation outcomes.

● Deliver CI/CD-ready software with unit/integration testing, code review and service-quality controls .

Mandatory Skills

● Strong Java (11/17) and Spring Boot; microservices and REST API design.

● Proven SaaS / multi-tenant application development experience.

● Strength across all SDLC phases with a shift-left quality mindset (automated testing, code review).

● Hands-on with at least one cloud platform (AWS, Azure or GCP).

● Familiarity with at least one AI / LLM platform: Azure AI Foundry, Google Vertex AI, OpenAI, Anthropic or AWS Bedrock.

● CI/CD pipelines and Git-based workflows (GitHub Actions, Jenkins, or equivalent)

● Hands-on telemetry / observability experience and strong integration / connector development.

● Streaming and event-driven pipelines - Kafka, GCP Pub/Sub, Azure Event Hub or AWS EventBridge/Kinesis.

● Working knowledge of cloud audit / log sources: AWS CloudTrail, GCP Cloud Audit Logs, Azure Monitor, Microsoft Sentinel

Preferred Skills

• Exposure to LLM evaluation concepts - hallucination, toxicity, bias/fairness, PII detection, robustness.

• Awareness of EU AI Act / Responsible AI frameworks.

• Python for tooling, dataset generation or evaluation scripts.

• Containerization and orchestration (Docker, Kubernetes); vector databases / RAG concepts.

• Observability and dashboarding tooling (Grafana, App Insights, OpenTelemetry, Datadog) and Python for tooling/automation.

• SIEM platforms - Microsoft Sentinel, Splunk, IBM QRadar or Elastic SIEM.