Key Responsibilities
- Agentic System Design: Design, develop, and deploy autonomous AI agents capable of complex, goal-oriented reasoning, planning, and task execution using modern agentic frameworks.
- RAG System Development: Build and optimize robust Retrieval-Augmented Generation (RAG) pipelines to ground LLMs in proprietary data, ensuring factual accuracy and data security.
- Tool Integration & Function Calling: Equip AI agents with the ability to use external APIs, databases, and custom tools to perform actions in the real world.
- Orchestration Frameworks: Master and implement core logic using orchestration tools like LangChain, LlamaIndex, LangGraph, or CrewAI to manage conversation state and agent workflows.
- Prompt Engineering & Alignment: Develop systematic Prompt Engineering strategies to maximize agent reliability and output quality.
- Production Deployment: Develop secure, low-latency API services (using Python/FastAPI) to serve LLM applications, ensuring high availability and scalability.
- Evaluation & Quality Assurance (Evals):Design and implement continuous evaluation frameworks to measure performance against business metrics. This includes developing ground truth datasets, implementing LLM-as-a-Judge scoring, and monitoring for hallucinations, factual correctness, and prompt injection.
Required Skills and Qualifications
- Programming:3+ years of professional software development experience, with expert proficiency in Python.
- LLM Application Experience: Direct experience building and deploying production-level applications using major LLM APIs or open-source models.
- Core RAG Expertise: Deep practical knowledge of designing, implementing, and optimizing a RAG system and proficiency with Vector Databases (e.g., Pinecone, Chroma, Milvus).
- Agent Orchestration: Proven experience with at least one major agent orchestration library (LangChain, LlamaIndex, or similar state machine/graph-based frameworks).
- Evaluation Tools: Hands-on experience with LLM evaluation frameworks like RAGAS, DeepEval, or LangSmith to create automated performance benchmarks.
- Back-end Engineering: Strong experience developing and maintaining RESTful APIs and integrating with various data sources (SQL/NoSQL).
- DevOps Fundamentals: Working knowledge of Docker, CI/CD pipeline and cloud infrastructure for deployment.
