Kafka Engineer/Administrator/SRE - 5+ Years of exp - Contract to Hire role

Role Description

This is a contract-to-hire role for a Kafka Engineer with 5+ years of experience. The Kafka Engineer will be responsible for designing, developing, and maintaining data pipelines using Apache Kafka to support real-time and batch processing requirements. The candidate will also collaborate with cross-functional teams to optimize system performance and scalability. This is a hybrid role located in Chennai, offering flexibility for some remote work.

Qualifications

Proficiency in languages like Python, Go, Java, or Bash forautomation scripts, tools, and integrations. Involves writing clean, maintainable code, debugging, API interactions, version control (e.g., Git), and unit testing.

In-depth knowledge of Linux/Unix. Includes managing processes, file systems, permissions, kernel tuning, shell scripting, server configuration, updates, and security.

Expertise in cloud platforms (AWS, GCP, Azure) and tools like Terraform or CloudFormation for infrastructure as code (IaC). Includes managing virtual machines, serverless architectures, and container orchestration (e.g., Kubernetes, ECS) for scalability and high availability.

Understanding of TCP/IP, HTTP, DNS, load balancing, VPNs, and firewalls. Includes configuring network services and troubleshooting with tools like Wireshark or traceroute.

Proficiency in tools like Splunk, Dynatrace, Prometheus, Grafana, Datadog, Jaeger/Zipkin for logs, metrics, and tracing. Involves defining Service Level Indicators (SLIs), setting Service Level Objectives (SLOs), and creating dashboards for system health insights.

Experience with CI/CD pipelines using Jenkins, GitLab CI, or GitHub Actions. Includes automating build, test, and deployment processes, as well as rollback mechanisms for reliability.

Skills in diagnosing and resolving production issues using logs, metrics, and debugging tools. Includes incident management (e.g., PagerDuty), root cause analysis (RCA), and blameless postmortems.

Familiarity with securing infrastructure, managing secrets (e.g., Vault), zero-trust principles, and compliance standards (e.g., SOC 2, GDPR). Includes vulnerability management and secure configuration.

Expertise in managing and operating Apache Kafka, NATS and MQ. Includes configuring topics (Kafka) or subjects (NATS), ensuring high availability, scaling clusters, monitoring performance metrics (e.g., consumer lag, throughput), and troubleshooting issues like message loss or latency. Involves understanding partitioning (Kafka) and pub/sub patterns (NATS) for event streaming and messaging.