Job Title: Lead Prompt Engineer

Employment Type: Full-time

Work Type: Hybrid

Duration: 12 months (Annual Renewal)

Location: Gurgaon or Hyderabad

ABOUT THE ENGAGEMENT:

Aceolution is hiring for the work that supports the development, evaluation, and safety calibration of frontier generative AI systems used by hundreds of millions of users worldwide.

KEY RESPONSIBILITIES:

As Per the engagement scope: architecture understanding, complex prompt design, and defining QA frameworks. You are the technical owner of the pod and the primary technical voice in client calibration sessions.

Own the architectural direction of the client tools API integration. Define how prompts, RAG, and evaluation infrastructure fit together for each workflow the pod supports.
Design complex prompt patterns. Set the standards for few-shot construction, negative constraints, and structured output that the team builds against.
Define and own the QA framework. Specify the Golden Dataset structure, regression-test methodology, and the rubric for good vs bad output.
Lead the adversarial testing approach. Ensure prompts are robust to malicious input, prompt injection, and edge-case content.
Review and approve all production prompt deployments. Last line of defense before a change reaches the live orchestration layer.
Conduct weekly architecture reviews with Prompt Engineers — direct, technical feedback.
Represent the team in client technical reviews. Defend architecture decisions and translate client product feedback into engineering work.
Drive the team's 95% alignment success metric, plus secondary metrics: false-positive rate, average handle time reduction, engineering-feedback acceptance rate.

MUST-HAVE SKILLS AND EXPERIENCE:

5–12 years total experience, with at least 3 years working hands-on with LLMs in production — not just inference, not just chat; production deployments with real users.
Demonstrable production experience with at least one major LLM API — Gemini, OpenAI, or Anthropic. Gemini-specific experience is strong-to-have but not mandatory; a strong candidate from another LLM platform will ramp in 2–3 weeks. Must be able to discuss context window optimization, token economics, and model-specific behavior quirks fluently.
Experience designing RAG systems end-to-end: chunking strategy, embedding model selection, vector DB choice, retrieval evaluation.
Experience designing or running adversarial-testing frameworks. You have stress-tested LLM systems against malicious or edge-case input.
Strong Python proficiency. You read and review code daily; you do not hand it off to others.
Experience translating ambiguous client requirements — where guidelines are 50% indicative, not 100% deterministic — into structured prompts and evaluation criteria.
Demonstrated stakeholder management at the Director / Senior PM level on the client side. You can defend an architecture decision in a room of senior client engineers.

STRONG-TO-HAVE SKILLS:

Experience with multi-agent frameworks: LangGraph, CrewAI, AutoGen, or custom orchestration.
Familiarity with eval frameworks: DeepEval, Ragas, OpenAI evals, or in-house equivalents.
Experience with model fine-tuning approaches — LoRA, QLoRA, full fine-tuning.
Background in content moderation, ad-tech evaluation, or trust-and-safety systems.
Experience working in a vendor or consulting model with a single high-stakes client.

Important notice:

Aceolution Inc. will never request a monetary deposit for any role or project with the company, and our recruitment and sourcing teams only use @aceolution.com address when emailing candidates. Ignore aceolutions.com which is a spammer email ID doing rounds over the past few months.