Machine Learning Engineer

Position: Machine Learning Engineer

Department: Artificial Intelligence / Machine Learning Engineering

Reports To: Lead AI Engineer

Location: Pune

Experience: Minimum 3 Years

Compensation: As per market standards

Job Overview:

We are seeking a highly analytical and data-driven Machine Learning Engineer to design, implement, and continuously improve evaluation frameworks and generation pipelines for AI-powered structured document generation systems.

The successful candidate will be responsible for building automated quality measurement systems, optimizing retrieval and generation performance, improving prompt engineering workflows, and ensuring output quality remains consistent as models, knowledge bases, and platform capabilities evolve. This role requires deep expertise in production LLM systems, Retrieval-Augmented Generation (RAG), evaluation methodologies, and statistical analysis.

Roles & Responsibilities:

LLM application engineering

•    Production LLM applications built and iterated — not fine-tuning experiments

•    Evaluation framework design — automated evaluation pipelines for LLM outputs, quality measurement, regression detection

•    Prompt engineering — systematic, experimental, data-driven. Test variants, measure results, draw conclusions from evidence.

•    RAG architecture — end-to-end understanding of how retrieval quality affects generation quality

Engineering and statistics

•    Python — expert level

•    Statistical thinking — evaluation design that measures what matters and cannot be gamed by easy metrics

•    Structured data generation — LLMs producing outputs in defined tabular formats rather than free text

•    At least one example of a production LLM system improved systematically — what was measured, what changed, what the outcome was

AI & Pipeline Knowledge - Required

•    Anthropic Claude API — tool use, system prompts, context window management, output reliability in production

•    Understanding of how context window composition affects generation quality — what to include, what to exclude, and why

•    Experience with evaluation frameworks for structured outputs — where standard NLP metrics fail and what to use instead

•    Prompt engineering as an engineering discipline — version control, A/B testing, systematic improvement cycles

What You Will Own:

•    Evaluation framework — automated measurement of output quality against domain-specific criteria. Accuracy rate, classification precision, rule coverage, overall quality score.

•    Generation pipeline optimisation — improving quality, consistency, and accuracy of structured outputs through prompt engineering, retrieval strategy, and knowledge base structuring

•    Embedding strategy — structuring source data for maximum retrieval quality in generation tasks

•    Regression detection — ensuring output quality improves or holds steady as the knowledge base and pipeline change

•    Prompt engineering at scale — systematic evaluation of prompt variants across real source documents, data-driven iteration

•    Knowledge base quality — identifying gaps and inconsistencies in embedded governance knowledge and programme-specific context

•    Model evaluation — systematic assessment of output quality as model versions change

8th June 2026

CT Automotive India