Job Summary:
We are seeking a Senior Prompt Engineer with 5+ years of experience in AI and LLM-driven application development. This role focuses on designing, testing, and optimizing enterprise-grade prompts for GenAI, RAG, and agentic systems. The ideal candidate will have hands-on experience with LLM APIs, vector databases, and multi-agent orchestration frameworks.
Strong expertise in prompt engineering, context design, and output control is essential.
Experience with Azure-based AI services, governance, security, and compliance is highly preferred. The role requires balancing accuracy, cost, performance and scalability in production AI systems.
In this Role, Your Responsibilities Will Be:
- Design, develop, test, and optimize advanced prompt strategies for LLMs (Open AI GPT, Claude, LLaMA, Mistral, Gemini) across enterprise use cases.
- Work closely with functional teams to engineer, test, and optimize prompts that translate business requirements into effective LLM-driven application behavior.
- Engineer multi-step prompt chains, system prompts, role-based prompts and meta-prompts for complex reasoning, planning, and decision-making tasks.
- Develop reusable prompt templates, prompt COT,ToT, Meta Prompting, Conversational Few-Shot and structured flows using frameworks like LangChain, LangGraph, and PromptLayer.
- Develop and maintain prompt orchestration pipelines for RAG, agentic AI, and multi-agent systems using LangChain, CrewAI, and MCP-based architectures.
- Engineer RAG-optimized prompts integrated with vector databases (PostreSQL, Redis,FAISS, Pinecone, Chroma) and re-ranking pipelines.
- Support agentic and multi-agent frameworks.
- Implement function-calling and schema-enforced prompting (JSON/YAML, tool/function APIs).
- Design hallucination-resistant, jailbreak-safe prompts with built-in guardrails and filters.
- Tune LLM parameters (temperature, top_p, penalties, max tokens) for accuracycostlatency balance.
- Develop reusable prompt templates, chains, and graphs (LangChain, LangGraph, PromptLayer).
- Perform A/B testing, regression testing, and prompt versioning for continuous optimization.
- Optimize token usage and context-window management to reduce inference cost.
- Align prompts with fine-tuning / LoRA / RLHF and embedding strategies.
- Implement PII/PHI masking, RBAC-aware prompts, and compliance-safe instructions (GDPR, HIPAA, ISO 27001).
- Monitor prompt drift, response instability, and output quality in production.
- Define and track prompt KPIs (quality, latency, cost, consistency).
- Ensure compliance with internal data governance and security policies.
Qualifications:
- Bachelor's or Master's degree in Computer Science, Data Science, Statistics, Mathematics, or a related field over 5+ years.
- Relevant Certifications/trainings on Gen AI, Prompt Engineering.
- Strong hands-on experience with LLM prompt engineering in production
- Experience integrating Prompts with RAG systems with vector databases.
- Hands-on Prompt Engineer with agentic frameworks and finetuning.
- Familiar with fine-tuning methods.
- Strong Python + API integration (FastAPI/Flask)
- Knowledge of LLM security, hallucination control, bias mitigation
- Experience/Knowledge in LLMOps/MLOps, MLflow, LangSmith, W&B, PromptLayer
- Understanding of PII protection, RBAC, compliance (GDPR/HIPAA/ISO)
- Excellent problem-solving, communication, and team collaboration skills.
Preferred Qualifications:
- Relevant certifications (Azure AI, Generative AI, LLMOps, Prompt Engineering)
- Experience with enterprise copilots, Chatbots, and agentic AI systems
- Hands-on with PromptOps / LLMOps governance and lifecycle management
- Expertise in semantic prompting, RAG, and context engineering
- Experience with LLM evaluation tools (RAGAS, TruLens, DeepEval, PromptLayer)
- Strong knowledge of LLM safety, guardrails and hallucination mitigation
- Familiar with hybrid model routing, fallback strategies and AutoPrompting
- Exposure to regulated domains and large-scale AI systems
- Certified inAzure AI Fundamentals (AI-900), Azure AI Engineer Associate (AI-102)
- Experience with big data technologies
