Principal Software Engineer in Test

Role summary

This is a hands-on engineering role focused on building reliable, scalable test systems for a distributed, event-driven platform. You will specialize in integration, end-to-end, and performance testing, with a strong emphasis on eliminating flakiness, improving signal in CI/CD, and ensuring systems are testable by design. You will work closely with product and platform engineers to design for testability, validate complex workflows, and leverage AI to accelerate test development, debugging, and coverage analysis. A good candidate will continuously challenge test strategy to maximize confidence while minimizing test suite cost and execution time.

Key responsibilities

Quality Strategy & Architecture

• Define and drive end-to-end test strategy across services, APIs, and workflows
• Establish scalable, reusable automation frameworks aligned with architecture patterns
• Promote quality as a shared responsibility across teams

Automation & Testing

• Partner with engineers during design and development to ensure systems are observable, debuggable, and testable
• Build and scale API, integration, and end-to-end automation using open-source tools
• Design test coverage for microservices and distributed systems
• Validate event-driven and asynchronous workflows, including failure and retry scenarios

Test Reliability & Signal

• Identify, debug, and eliminate flaky tests across pipelines
• Design and manage test data strategies for complex, stateful systems
• Improve determinism and stability of integration and E2E test suites
• Increase signal-to-noise ratio in CI by reducing false positives/negatives
• Build tooling and practices for test isolation, data management, and environment consistency

Performance Engineering

• Develop and execute performance and load testing strategies (Locust, k6, JMeter)
• Model realistic production traffic and identify system bottlenecks
• Partner with engineering teams to optimize system performance

CI/CD & Shift-Left Enablement

• Integrate automated testing into CI/CD pipelines for fast, reliable feedback
• Drive early testing practices across the development lifecycle
• Ensure stability and reliability of automated test execution

Observability & Debugging

• Leverage logs, metrics, and traces to validate system behavior
• Use observability tools (e.g., Grafana) to diagnose issues and improve reliability

Required qualifications (Hard requirements)

• 12–14 years in QA Automation / Quality Engineering
• Strong programming experience in C#/.NET Core, Python, or Java
• Demonstrated ability to use AI tools to significantly accelerate development, debugging, and test creation
• Ability to integrate AI into testing workflows (test generation, log analysis, coverage gaps)
• Deep experience in API automation and distributed systems testing
• Hands-on experience with microservices and event-driven architecture (Kafka or similar)
• Experience with performance testing tools (Locust, k6, JMeter)
• Strong experience integrating testing into CI/CD pipelines
• Familiarity with observability practices (logs, metrics, traces; Grafana preferred)
• Strong collaboration and communication skills across cross-functional teams

Preferred qualifications

• Bachelor's or master's degree in computer science, engineering or related field
• Experience with AI-assisted testing or generative AI tools
• Exposure to test architecture design at scale
• Experience influencing quality practices across multiple teams

What success looks like

• Measurable reduction in escaped defects and improved system reliability
• Significant reduction in flaky test rates
• Faster and more reliable CI/CD feedback cycles
• Improved visibility into system performance and behavior
• Strong adoption of automation and quality practices across teams

Why join us

You'll work on large-scale platform modernization with real impact on how millions of consumers manage healthcare spending. This role offers the opportunity to shape quality engineering practices, work with modern architectures, and drive meaningful engineering outcomes.