This role is for one of our clients

We are seeking a Software Engineer – Machine Learning Platform to build and scale an AI-native platform that powers intelligent, data-driven applications across complex enterprise workflows. In this role, you will transform unified operational data into real-time intelligence by building the infrastructure that enables context-aware AI agents, retrieval systems, and production-grade machine learning models.

You will own core platform capabilities including the LLM control plane, model gateway, agent runtime, retrieval infrastructure, and ML platform services. Your work will establish the standards for building, evaluating, deploying, and governing both generative AI and traditional machine learning solutions, enabling product teams to deliver AI-powered features safely, reliably, and at scale.

Requirements

Key Responsibilities

LLM Platform & AI Infrastructure

Design and operate an enterprise-grade LLM gateway and control plane featuring intelligent routing, rate limiting, failover, token management, and cost optimization.
Develop unified APIs and SDKs (REST/gRPC) with standardized schemas, structured outputs, caching, and comprehensive observability.
Implement security and privacy controls including content moderation, prompt validation, response filtering, and PII protection.
Enable seamless multi-model and multi-provider LLM integration with automated versioning, canary deployments, and rollback mechanisms.

Agent Platform & Orchestration

Build and maintain the runtime environment for AI agents, including tool registries, permissions, function calling, retrieval, and grounding capabilities.
Design orchestration patterns for sequential, planner-executor, streaming, and long-running workflows.
Integrate human-in-the-loop approval mechanisms and safe execution controls before automated actions interact with production systems.
Monitor agent performance, reliability, and operational health through telemetry and evaluation frameworks.

Machine Learning Platform

Develop scalable infrastructure supporting training, deployment, and inference for classical machine learning models and deep learning workloads.
Build standardized pipelines for experiment tracking, model packaging, deployment, and lifecycle management.
Implement automated monitoring for model performance, drift detection, retraining, and continuous optimization.
Support online and batch inference workflows for production ML applications.

Knowledge Graph & Data Platform

Build and evolve knowledge graph infrastructure and entity resolution systems.
Develop reliable, scalable data ingestion and transformation pipelines.
Deliver contextual information to AI systems through secure access controls, lineage tracking, and governance.
Implement hybrid retrieval systems combining graph databases, vector search, and keyword search for high-quality context retrieval.

Model Governance & Reliability

Design continuous evaluation frameworks measuring model quality, factual accuracy, bias, safety, and performance.
Define and monitor Service Level Objectives (SLOs) for latency, uptime, throughput, and infrastructure cost.
Manage model registries, versioning, approvals, audit trails, and reproducible deployment workflows.
Build autoscaling strategies, monitoring dashboards, and cost optimization mechanisms.

Developer Experience

Create reusable SDKs, templates, developer tools, CLIs, documentation, and sandbox environments.
Improve platform usability by establishing best practices for ML engineering, MLOps, AI safety, and deployment workflows.
Mentor engineering teams and promote scalable engineering standards across the organization.

Required Qualifications

5–11+ years of experience building large-scale machine learning platforms, data platforms, or distributed software systems.
Strong software engineering skills with expertise in distributed systems, concurrency, and scalable API design.
Production experience with Python and at least one of Java, Go, or Scala.
Hands-on experience building microservices and REST/gRPC APIs.
Strong understanding of cloud-native architecture, containers, and Kubernetes.
Experience with MLOps platforms including ML pipelines, experiment tracking, model registries, CI/CD, A/B testing, shadow deployments, and feature engineering workflows.
Practical experience deploying and operating production machine learning systems.
Experience building or managing LLM infrastructure, including routing, provider abstraction, caching, quota management, and cost optimization.
Knowledge of AI agent architectures, orchestration frameworks, tool calling, safety mechanisms, and online evaluation techniques.
Experience with knowledge graphs, vector databases, GraphQL, and hybrid retrieval architectures.

Preferred Qualifications

Experience with Airflow, Kubeflow, MLflow, Spark, Flink, Kafka, or similar ML infrastructure tools.
Strong cloud experience on AWS or equivalent cloud platforms.
Familiarity with vector databases such as pgvector, Milvus, Qdrant, or similar technologies.
Experience with graph databases including Neo4j, Amazon Neptune, or TigerGraph.
Background in retrieval-augmented generation (RAG), AI agents, and enterprise AI platforms.
Experience building multi-tenant SaaS infrastructure with strong observability and governance.

Ideal Candidate Profile

The ideal candidate:

Thinks of infrastructure as a product and prioritizes developer experience.
Designs systems with reliability, observability, scalability, and security from the outset.
Is passionate about AI platforms, LLMs, machine learning infrastructure, and agentic systems.
Balances performance, latency, reliability, and operational cost when designing solutions.
Prefers vendor-agnostic architectures that maximize flexibility and portability.
Enjoys documenting complex systems and mentoring engineering teams.

Why Join Us

Join a team building next-generation AI and machine learning infrastructure that powers intelligent enterprise applications at scale. You'll have the opportunity to shape platform architecture, enable AI innovation across multiple product teams, and build systems that combine classical machine learning with cutting-edge generative AI technologies.

Must-Have Skills

Python
Machine Learning Platform
MLOps
Distributed Systems
Kubernetes
REST APIs
AI/LLM Infrastructure

Good-to-Have Skills

ML Infrastructure
Java
Graph Databases
Vector Databases
RAG
AI Agents
Kafka
Spark
Kubeflow
MLflow

Software Engineer (Machine learning Platform)

Software Engineer - (Machine learning Platform)