We are looking for a Staff Engineer-Real-time Data Processing to design and develop highly scalable, low-latency data streaming platforms and processing engines. This role is ideal for engineers who enjoy building core systems and infrastructure that enable mission-critical analytics at scale. you'll work on solving some of the toughest data engineering challenges in healthcare.
A Day in the Life
- Architect, build, and maintain a large-scale real-time data processing platform.
- Collaborate with data scientists, product managers, and engineering teams to define system architecture and design.
- Optimize systems for scalability, reliability, and low-latency performance.
- Implement robust monitoring, alerting, and failover mechanisms to ensure high availability.
- Evaluate and integrate open-source and third-party streaming frameworks.
- Contribute to the overall engineering strategy and promote best practices for stream and event processing.
- Mentor junior engineers and lead technical initiatives.
What You Need
- 8+ years of experience in backend or data engineering roles, with a strong focus on building real-time systems or platforms.
- Hands-on experience with stream processing frameworks like Apache Flink, Apache Kafka Streams, or Apache Spark Streaming.
- Proficiency in Java, Scala, or Python or Go for building high-performance services.
- Strong understanding of distributed systems, event-driven architecture, and microservices.
- Experience with Kafka, Pulsar, or other distributed messaging systems.
- Working knowledge of containerization tools like Docker and orchestration tools like Kubernetes.
- Proficiency in observability tools such as Prometheus, Grafana, OpenTelemetry.
- Experience with cloud-native architectures and services (AWS, GCP, or Azure).
- Bachelors or masters degree in Computer Science, Engineering, or a related field.
