This role sits at the intersection of GenAI and pharmaceutical learning u2014 a rare opportunity to own AI avatar behavior in a highly regulated, high-stakes environment.

The Conversational AI Platform Owner is the single point of operational accountability for the LillyAvatarAssistant system. The role spans four connected disciplines: prompt and configuration management across the full prompt suite (Conversation, Evaluation, Compliance, and Drug Information) LLM model configuration and versioning publishing and release operations through the AI Avatar Publishing Platform and production support including incident response and rollback execution.

This is not a backend development role. It requires technical depth to manage prompts, model parameters, multi-language configurations, compliance-sensitive scoring logic, and structured release processes u2014 while operating within a validated, change-controlled pharmaceutical environment.

Key Responsibilities

1. Prompt Suite Ownership

Own and maintain the full prompt suite u2014 Conversation Prompts, Evaluation Prompts, Compliance Prompts, and Drug Information Prompts u2014 including their associated criteria and scoring logic
Maintain persona definitions, system messages, tone alignment, and behavioral reliability across all avatar configurations

2. LLM Model Configuration & Versioning

Manage LLM model configurations including inference parameters (temperature, top-p, max tokens), safety/guardrail settings, and content filter configurations
Maintain a model inventory (u201Cmodel cardsu201D) documenting model name/version, primary use, key settings, known limitations, and failure modes
Follow a disciplined model/prompt change process: define intent, curate golden test sets, validate, document evidence, confirm rollback path

3. Publishing Platform Operations

Create, edit, test, publish, and unpublish Avatars through the full Publishing Platform lifecycle
Manage Avatar Characteristics, Site Content, prompt configurations, and Test & Review workflows
Manage Drafts and Published Avatars pages including edit, test, publish/unpublish operations

4. Testing & Validation

Execute layered testing: developer checks, integration tests, LLM evaluation harnesses, manual smoke tests, and full UAT regression (per the established UAT checklist)
Develop and maintain golden conversation test sets for behavioral validation after model or prompt changes
Collect and document evidence: screenshots, transcript snippets, before/after comparisons, CI run links
Run UAT regression before releases or major configuration changes

5. Release Discipline & Stakeholder Sign-Offs

Drive the stage-gated release process: implement in non-prod, smoke test, deploy to draft, execute UAT, collect sign-offs, promote to production
Coordinate Business Stakeholder and System Owner acknowledgements per the established sign-off framework
Ensure change request documentation and evidence packages are complete per change management policy
Own post-release monitoring: confirm error rate, latency, and key user journeys are within expected thresholds

6. Production Support & Incident Response

Triage production issues using observability tools (logs, traces, metrics) to isolate avatar session or evaluation failures
Execute emergency response: unpublish problematic Avatars, coordinate platform-wide rollbacks with Development Leads
Document incident timeline, root cause, and preventive actions
Maintain a list of u201Cknown-goodu201D Avatars for regression testing and track platform dependencies (LLM provider status, auth/SSO, email service)

7. Documentation & Governance

Maintain prompt libraries, model cards, runbooks, architecture references, and release checklist templates
Follow governance practices including approvals, change tracking, and audit trails
Capture knowledge artifacts that reduce bus factor: architecture diagrams, data flows, monitoring dashboard links, alerting thresholds

8. Cross-Functional Collaboration

Partner with Product Owner, Tech Lead, Program Management, Development Leads, and vendors through iteration and release cycles
Support issue resolution by interpreting API responses and troubleshooting configuration and publishing errors in collaboration with Tech teams
Understand how avatar configurations connect to backend data stores (e.g., Azure Cosmos DB) and how API endpoints are managed (e.g., Azure APIM) to support troubleshooting and change impact assessment

Required Skills

Prompt & Content

Must Have

Strong writing and editing skills for AI avatar copy (dialogue, tone, persona, system messages)

Solid understanding of prompt engineering fundamentals: tone control, accuracy, consistency, behavioral alignment, and prompt layering conventions (system prompt u2192 avatar-level u2192 turn-level)

Ability to own evaluation, compliance, and drug information prompt logic u2014 including criteria management and scoring alignment

Familiarity with conversational design principles and voice/tone systems

Experience writing structured content for chat-based experiences (assistants, guided flows, role-play personas)

Familiarity with localization workflows: managing translation content, pronunciation guidance, and locale-specific access control across multiple languages

Platform & Operations

Must Have

Comfort executing UAT cycles and validating outputs using structured test cases and edge-case thinking

Experience with production support processes: incident triage, severity assessment, escalation paths, rollback execution, and root cause documentation

Comfort navigating observability and monitoring tools (e.g., Splunk, Datadog, App Insights, Grafana) to locate logs/traces and diagnose production issues

Understanding of release discipline: stage-gated deployments, evidence collection, stakeholder sign-off coordination, and change management compliance

Exposure to compliance-oriented content workflows and approval processes in regulated environments

Technical Foundations

Must Have

Working knowledge of LLM inference settings (temperature, top-p, max tokens, frequency/presence penalties) and their behavioral impact on conversation quality and scoring accuracy

Strong working knowledge of Markdown and HTML fundamentals for publishing, content rendering, and documentation formatting

Comfort working with structured configurations (JSON/YAML) and API-based workflows (e.g., Postman, cURL)

Understanding of versioning principles: change logs, audit trails, rollback procedures, and configuration version management

Understanding of role-based access control principles as applied to publishing platforms and content governance

High attention to detail with ability to prevent regressions and unintended behavior shifts

Nice to Have

Experience working with Conversational AI platforms (chatbots, virtual assistants, AI avatars, or agent-based systems)
Experience with LLM output evaluation methods: golden-set testing, before/after behavioral comparison, and scoring validation using evaluation harnesses
Basic understanding of Retrieval-Augmented Generation (RAG) concepts including document ingestion, embeddings, and search behavior
Exposure to cloud-based data integrations (Azure preferred) willingness to learn platform-specific configurations
Experience with pharmaceutical or healthcare compliance frameworks

Education & Qualifications

Required

Bacheloru2019s degree in Computer Science, Information Technology, Computational Linguistics, Human-Computer Interaction, Digital Media, or a related technical/interdisciplinary field
7u201310 years of professional experience across one or more of: conversational AI, prompt engineering, NLP/NLU platforms, content operations for AI systems, or technical product/platform operations

Preferred

Masteru2019s degree in AI/Machine Learning, Computational Linguistics, Cognitive Science, Learning Technologies, or a related advanced discipline
Professional certifications in cloud platforms (e.g., Azure AI Fundamentals, Azure AI Engineer Associate) or AI/ML (e.g., Google Professional ML Engineer, AWS Machine Learning Specialty)
Formal training or certification in prompt engineering, conversational design, or LLM application development
ITIL or similar certification demonstrating familiarity with service management, change control, and incident management frameworks

Lilly is dedicated to helping individuals with disabilities to actively engage in the workforce, ensuring equal opportunities when vying for positions. If you require accommodation to submit a resume for a position at Lilly, please complete the accommodation request form () for further assistance. Please note this is for individuals to request an accommodation as part of the application process and any other correspondence will not receive a response.

Lillyu00A0does not discriminate on the basis of age, race, color, religion, gender, sexual orientation, gender identity, gender expression, national origin, protected veteran status, disability or any other legally protected status.

#WeAreLilly

Jr. Data Engineer

Senior Data Engineer-Digital Banking Kotak 811-Regional Sales

Developer

Conversational AI Platform Owner

Systems Operations Engineer

Account Manager - Customer Success

Technical Support Engineer-II