Job Description:

We are looking for an experienced Kafka Administrator to manage and support enterprise Kafka environments including cluster configuration topic and partition management broker administration consumer lag monitoring security performance tuning upgrades and production troubleshooting
The ideal candidate will have hands on experience with Apache Kafka KRaft Linux monitoring tools automation frameworks and cloud infrastructure ensuring highly available and reliable messaging platforms

Key Responsibilities:

Provision configure and manage Kafka clusters Apache Kafka KRaft or ZooKeeper backed if legacy Confluent Platform components Schema Registry Kafka Connect ksqlDB REST Proxy Control Center and Confluent Cloud resources
Ensure high availability HA and resilience across multi AZ region deployments manage partition replication min
insync
replicas ISR and rack awareness
Implement and maintain client quotas broker quotas throttling and back pressure strategies
Plan and execute upgrades patching with zero minimal downtime maintain version currency against EOL and security advisories
Own backup restore workflows for metadata e
g
cluster configs Schema Registry and Connect configs
Implement authentication mTLS SASL SCRAM OAUTHBEARER with IdP or Kerberos if legacy and authorization Kafka ACLs Confluent RBAC
Enforce encryption in transit and at rest secret management Vault AKV SSM secure endpoint exposure and private connectivity VPC peering PrivateLink
Govern topic naming conventions retention policies schema evolution rules backward fully compatible and PII handling
Maintain audit trails broker Schema Registry Control Center Confluent Cloud audit logs and compliance artifacts SOX GDPR HIPAA PCI as applicable
Coordinate vulnerability management CVE watch hardening baselines CIS STIG alignment
Deploy and maintain metrics pipelines JMX Prometheus Grafana logs ELK OPensearch and tracing OpenTelemetry where applicable
Define and monitor SLOs SLIs produce consume latency end to end lag broker CPU heap file handles network throughput GC pauses
Analyze broker hotspots partition skew large message handling compaction and retention settings and tune GC JVM for sustained throughput
Implement consumer lag monitoring and alerting with actionable runbooks to prevent data loss or SLA breaches
Model workload growth topic partition scaling storage IOPS network egress ingress and broker sizing
For Confluent Cloud optimize environment cluster topic hierarchy throughput tiers retention costs and data flow egress right size Dedicated vs
Standard clusters
Design partitioning strategies to meet throughput targets while balancing consumer concurrency and avoiding small partitions anti patterns
Automate cluster and topic lifecycle using Terraform providers for Kafka Confluent Helm GitOps workflows

Technical Requirements:

2 5 years in platform SRE DevOps 2 years dedicated to Kafka Confluent admin in production
Strong hands on with Apache Kafka internals brokers controllers KRaft partitions replication log segments compaction retention
Confluent Platform Schema Registry Kafka Connect ksqlDB Control Center Confluent REST APIs or Confluent Cloud admin environments clusters RBAC audit logs networking
Proficiency with Kafka CLI tools and AdminClient usage deep knowledge of ACLs quotas transactions idempotent producers and exactly once semantics scenarios
Prometheus Grafana JMX exporters alerting systems Alertmanager etc
JVM tuning Linux performance NUMA file descriptors page cache disk RAID XFS and network tuning TCP buffers MTU jumbo frames where applicable
Linux administration systemd journald kernel params virtualization containers Docker Kubernetes
Cloud networking VPC VNET peering transit gateways PrivateLink Private Service Connect multi region architectures
Preferrable
Confluent Certified Administrator for Apache Kafka CCAAK CKA CKAD HashiCorp Terraform Associate major cloud certs Azure AWS GCP
Experience with Confluent for Kubernetes CFK for operator based deployments

Additional Responsibilities:

Infosys is a global leader in next generation digital services and consulting
We enable clients in more than 50 countries to navigate their digital transformation
With over four decades of experience in managing the systems and workings of global enterprises we expertly steer our clients through their digital journey
We do it by enabling the enterprise with an AI powered core that helps prioritize the execution of change
We also empower the business with agile digital at scale to deliver unprecedented levels of performance and customer delight
Our always on learning agenda drives their continuous improvement through building and transferring digital skills expertise and ideas from our innovation ecosystem
Infosys provides equal employment opportunities to applicants and employees without regard to race color sex gender identity sexual orientation religious practices and observances national origin pregnancy childbirth or related medical conditions status as a protected veteran or spouse family member of a protected veteran or disability

Preferred Skills:

Technology->Integration->Confluent Kafka,Technology->Cloud Platform->AWS Data Analytics->Amazon Managed Streaming for Apache Kafka

Kafka Admin