Job Description

Own the end-to-end technical architecture for a testing agentic AI platform, including multi-agent orchestration, tool interfaces, safety, observability, and scalable deployment.

Responsibilities

Define multi-agent system architecture (planner, executor, memory, tool adapters) and governance
Design tool-use layer (APIs, search, code, test frameworks) with secure sandboxing, enabling agentic systems to operate via tools and feedback loops
Establish evaluation and safety architecture (guardrails, red-teaming, policy engines) for non-deterministic behavior with human-in-the-loop review
Select model strategy (proprietary vs OSS), vector/store strategy, and retrieval patterns
Drive platform non-functionals: reliability, latency, cost, observability, and data privacy
Collaborate with PO/AI Lead on roadmap and technical debt; work with product teams to design, test, and deploy AI capabilities

Technical Skills

LLM/Agent Frameworks: LangChain, LlamaIndex, multi-agent orchestration, tool-calling, function calling
Retrieval & Memory: Vector databases (FAISS, Milvus, pgvector), embeddings
Evaluation/LLMOps: Prompt versioning, offline/online evaluations, MLflow, Weights & Biases (W&B), feature stores
Systems: Distributed services, event-driven architecture, gRPC/REST APIs, Kubernetes, Terraform, observability
Security & Compliance: Data governance, PII handling, secrets management, guardrails

Domain Skills