- Posted 2 hours ago
- Be among the first 10 applicants
Job Description
Langfuse Implementation & LLM Observability Engineer
Experience:- 6 to 10 Years
Objective
Implement an enterprise-grade Langfuse platform for monitoring LLM applications, configure observability, define and implement KPIs, and build dashboards for technical and business stakeholders.
Mandatory Skills
Hands-on experience implementing Langfuse in production environments.
Strong Python development.
Experience integrating LLM frameworks such as LangChain, LlamaIndex, OpenAI SDK, or similar.
Docker and Kubernetes deployment experience.
PostgreSQL administration and optimization.
REST APIs and microservices.
Authentication (SSO/OAuth/OIDC), RBAC, and secrets management.
Git, CI/CD, and DevOps practices.
Experience with cloud platforms (Azure preferred).
Preferred Skills
LLM evaluation frameworks (Giskard, Ragas, DeepEval, or similar).
Azure Monitor/Application Insights.
Vector databases and RAG architectures.
Knowledge of AI governance and Responsible AI.
Responsibilities
Deploy and configure Langfuse.
Integrate Langfuse SDK into LLM applications.
Configure environments, users, projects, prompts, and datasets.
Enable tracing, prompt versioning, sessions, and evaluations.
Configure token usage and cost tracking.
Implement custom metrics and business KPIs.
Build executive and operational dashboards.
Configure alerts for latency, failures, cost spikes, and quality degradation.
Integrate evaluation frameworks for hallucination, faithfulness, and relevance.
Prepare documentation, runbooks, and knowledge transfer.
Support UAT and production rollout.
Built Dashboard KPIs
Deliverables
Production-ready Langfuse deployment.
Complete Langfuse configuration.
KPI definitions and implementation.
Executive dashboard.
Operations dashboard.
Cost dashboard.
Quality dashboard.
Alert configuration.
Documentation and architecture.
Knowledge transfer sessions.
Production handover.
More Info
Key Skills
About Company
