The Opportunity
- Adecco is partnering our client, a famous Tech MNC
- We are looking for a Senior Software Engineer
- The role will start out as a 6-months contract (renewable)
- Candidates who are immediately available/ able to start work within short notice will be preferred
The Talent
- Minimum of 2 years experience in or at at least most:
- EvalsHub, LangSmith, LangGraph/LangChain, FastAPI, Temporal, Grafana, Redis, or similar technologies.
- React/TypeScript and internal developer-tooling interfaces.
- LLM-as-judge, multi-turn evaluation, tool/MCP evaluation, and agent-trajectory analysis.
- Text-to-SQL/Text-to-DSL, RAG, semantic retrieval, embedding evaluation, or model benchmarking.
- Building platform capabilities that can be reused across several AI products.
- Strong analytical and problem-solving skills related to geo data
- Excellent project management and cross-department communication abilities
- Detail-oriented and metrics-driven approach to continuous improvement
Job Description
- Embed with AI product teams and own measurable agent-quality outcomes.
- Build and maintain golden datasets, regression suites, evaluation pipelines, and quality dashboards.
- Convert product rules and human review procedures into deterministic, numeric, LLM-as-judge, trajectory, tool-selection, SOP-adherence, and multi-turn evaluators.
- Diagnose evaluation and production-trace failures, then convert findings into dataset improvements, evaluator changes, or fixes to prompts, tools, skills, and agent workflows.
- Instrument distributed agent systems using OpenTelemetry, including Temporal workflows and Go/Python services.
- Ensure traces contain consistent inputs, outputs, tool calls, metadata, feedback, and root-agent results.
- Develop reusable capabilities across EvalsHub backend, SDK, CLI/plugins, and supporting frontend interfaces.
- Integrate evaluations into CI/CD and model-release workflows.
- Run model, retrieval, embedding, and agent-architecture experiments balance accuracy, latency, and cost.
- Drive migrations from legacy evaluation and observability systems.
- Write technical designs and documentation, run demonstrations, and enable engineering teams to use evaluation tooling independently.
- Participate in the LLMOps operational rotation and support production-quality integrations.
Next Step
- Prepare your updated resume
- Send your resume to [Confidential Information]
- All shortlisted candidates will be contacted
Liu XinYang
EA Licence Number: 91C2918
Personnel Registration Number: R1988872
