Gen AI Devloper/Tester

Role Summary

• We are looking for a GenAI Developer Tester who can design build and validate enterprise grade Retrieval Augmented Generation RAG Agents Agentic AI and generative AI applications on Azure This hybrid role blends hands-on development with rigorous testing and evaluation to ensure reliable grounded and production ready solutions

Responsibilities

• Build RAG bots Agentic AI bots and GenAI applications using Azure OpenAI Azure AI Search Python Flask Azure Functions HTTP Queue BlobTimer and APIM with secure configuration via environment variables
• Implement scalable data layers using Cosmos DB and Azure Storage Blob Queue Table optimize chunking embeddings and hybrid vector search pipelines
• Lead prompt engineering and tuning across system prompts retrieval prompts and tool use prompts design guardrails to minimize hallucinations and improve grounding
• Establish automated testing with pytest and RAG evaluation using Ragas faithfulness precision recall answer correctness
• Ensure observability with Application Insights advanced logging and end to end traceability
• Orchestrate workflows with LangChain and optionally LangGraph for stateful agent flows
• Deliver production SLAs by tuning rate limits scaling AI Search tiers and optimizing cost performance

Must Have Skills

• Strong in Python FlaskFastAPI Azure Functions APIM Azure AI Search Azure OpenAI Cosmos DB and Azure Storage
• Proven experience with prompt engineering and tuning
• Test automation with pytest evaluation with Ragas knowledge of grounding techniques hybrid search semantic ranking context filtering

Nice to Have

• Experience with LangGraph for agentic flows
• CICD pipelines Azure DevOpsGitHub Actions
• Structured output validation and finetuning exposure