Lead Software Engineer

We're looking for a platform engineer to own the Kubernetes platform behind Versa Messaging Service (VMS), an Ubuntu-based platform delivered to customer environments as an appliance, from single-node to multi-node HA deployments. You'll make the cluster reliable, secure and upgradeable. Microservices run on top of it, and you'll partner with service owners when platform changes affect them.

What You Will Do

Cluster lifecycle

• Build, operate and upgrade kubeadm-managed clusters on Ubuntu LTS, covering single-node and multi-node HA topologies.
• Own installation, scale-out (single to multi-node) and upgrade automation, and validate that services stay healthy through upgrade windows.
• Manage etcd operations: backup/restore, defragmentation and disaster recovery.
• Own cluster networking (CNI, CoreDNS, load balancing and ingress appropriate to each topology) and the namespace/RBAC patterns used by VMS workloads.
• Maintain Kubernetes manifests, Helm/Kustomize packaging, and platform configuration.

Observability, security and reliability

• Run and tune Prometheus, Grafana and Elastic-based logging, with alerts that separate cluster symptoms from workload symptoms.
• Apply appliance hardening, including Ubuntu FIPS/STIG/CIS patterns, ingress hardening (WAF/ModSecurity), Pod Security Standards and NetworkPolicy, without compromising upgradeability or availability.

Incidents and partnership

• Lead diagnosis of control-plane, networking/DNS, storage, resource-pressure and upgrade-regression incidents.
• Write runbooks, mentor engineers, and help set platform release and upgrade policy.

Required Qualifications

• 5-12 years building and operating production infrastructure, with a clear majority in hands-on Kubernetes and Linux.
• Deep Kubernetes experience on bare metal or on-prem: kubeadm init/join/upgrade, kubelet, CNI and DNS troubleshooting, RBAC, Secrets, rolling updates.
• Experience running both single-node and multi-node/HA clusters, including HA control planes, etcd quorum and failure modes, and single-node upgrade strategies.
• Strong Ubuntu/Linux skills: systemd, networking, firewalls, routing, TLS/PKI and SANs.
• Experience with Debian/Ubuntu packaging (dpkg/apt, pre/post-install hooks) and upgrade orchestration alongside application releases.
• Container runtime expertise with containerd (Docker for image builds), including image loading for air-gapped environments.
• Strong Bash plus Python (or Go) for installation, upgrade and maintenance automation.
• Storage fundamentals: CSI, local volumes and StatefulSet behavior.
• Infrastructure-level experience with Kafka or a similar message bus (connectivity, TLS, failure modes).
• Sound judgment on change risk: upgrade ordering, certificate-rotation windows, node drains, and when to block a release.

Nice to Have

• Appliance or constrained-environment Kubernetes (ISO/OVA image builds, multi-tenancy on one cluster).
• Experience with MetalLB/L2, kube-vip or Cilium/Calico, and CNI recovery.
• CI/CD with upgrade test matrices, and supply chain practices (image signing, SBOMs, CVE scanning).
• SLO-oriented alerting; security or networking product background.