Quality Engineering Lead

Optum is a global organization that delivers care, aided by technology to help millions of people live healthier lives. The work you do with our team will directly improve health outcomes by connecting people with the care, pharmacy benefits, data and resources they need to feel their best. Here, you will find a culture guided by inclusion, talented peers, comprehensive benefits and career development opportunities. Come make an impact on the communities we serve as you help us advance health optimization on a global scale. Join us to start Caring. Connecting. Growing together.

Job Summary

We are seeking a highly skilled Software Development Engineer in Test (SDET) with strong experience in Python-based test automation and hands-on knowledge of LLM (Large Language Model) development, testing, or evaluation.

Key Responsibilities

• Lead in-depth research on automation frameworks and define best practices.

• Design, develop, and maintain robust Python-based automation frameworks for backend, UI, API, and AI-model workflows.

• Create automated test suites for LLM-driven systems, including prompt evaluation, output validation, and AI behavioral consistency testing.

• Implement strategies for LLM latency, accuracy, and reliability validation using Python concurrency patterns.

• Integrate automated testing into CI/CD pipelines and optimize execution for model services.

• Collaborate with development, data science, and MLOps teams to ensure safe and correct model behavior.

• Develop advanced automation scripts for web, mobile, and API testing.

• Collaborate with Agile, DevOps, and development teams to resolve complex issues.

• Perform R&D on testing technologies and oversee POCs.

• Maintain documentation and ensure compliance with automation standards.

• Debug issues related to model inference, data pipelines, and automation frameworks.

• Participate in Agile ceremonies and continuous improvement.

Required Qualifications:-

• 5+ years of experience in software testing, automation, or development.
• Strong Python experience for automation
• Hands-on experience testing or building LLM systems.
• Experience designing or maintaining automation frameworks.
• Solid knowledge of CI/CD pipelines and DevOps practices.
• Strong debugging and problemsolving skills.

Bonus / Nice-to-Have Skills

• Experience with LLM evaluation frameworks (Ragas, DeepEval).

• Prompt engineering or model finetuning experience.

• Experience with Postman, BDD tools, or Katalon.

• Familiarity with cloud-based model hosting.

Soft Skills

• Excellent communication and collaboration skills.

• Proactive, self-driven, and comfortable working independently or cross-functionally.

Preferred Qualifications:

• Bachelor's degree in Computer Science, Engineering, or a related field.

• Experience in the healthcare domain is a strong plus.

• Familiarity with accessibility testing frameworks and standards.

• Exposure to cloud-based testing platforms and mobile device farms.

#AIForBetterHealthcare