Data Streaming Engineer

Job Description – Data Streaming Engineer / Kafka & Streaming Specialist 
Role Overview 
We are looking for a skilled Data Streaming Engineer with Handson experience in Confluent Kafka, real-time streaming platforms such as Apache Flink, and DevOps practices involving Git and Azure DevOps.  

The ideal candidate will work on building, managing, and optimizing large‑scale event‑driven data pipelines across enterprise systems. 

Key Responsibilities 

 Design, create, and manage Kafka topics, partitions, consumer groups, and ACLs. 
Configure, deploy, and support Kafka Connect connectors (Source & Sink). 
Develop, maintain, and troubleshoot real‑time streaming applications using platforms like Apache Flink. 
Monitor streaming pipelines for performance, reliability, and throughput. 
Work closely with cross‑functional teams to ensure seamless integration of streaming components across environments. 
Use Git for version control and follow best practices for branching, pull requests, and code reviews. 
Manage CI/CD workflows and pipelines using Azure DevOps. 
Participate in performance tuning, capacity planning, and production support for streaming workloads. 
Document architecture, flows, and operational procedures. 

Required Technical Skills 
Strong experience in Confluent Kafka ecosystem including: 

 Topics, partitions, retention, replication 
Consumer groups, offsets, rebalancing 
Kafka Connect (Source/Sink connectors) 
Schema Registry and Avro/JSON schemas 

Experience building or supporting streaming pipelines using Apache Flink or similar frameworks. 
Working knowledge of Git (branching strategy, merges, PR reviews). 
Exposure to Azure DevOps for build pipelines, release pipelines, and repo management. 

Experience with CI/CD automation, containerization, or YAML-based pipeline configurations. 

Soft Skills 

 Strong analytical and debugging skills. 
Excellent communication and cross-team collaboration abilities. 
Ability to work in Agile environments with fast iteration cycles. 
Ownership mindset with focus on reliability and operational excellence. 

Skill Matrix 

Skill or Category 

Mandatory/Non-Mandatory 

Evaluation Focus 

Minimum Rating Scale (0-5) 

Kubernetes/Kibana/Grafana 

Non-Mandatory 

Knowledge must be preferred 

2 

Google Cloud Platform Services 

Mandatory 

Hands on in GCP platform 

2 

Infrastructure as Code preferred Terraform 

Non-Mandatory 

Hands on experience in Terraform 

2 

Apache Flink Platform 

Mandatory 

Knowledge on Streaming platform 

4 

Deployment Pipeline Automation and Execution using Azure Devops 

Mandatory 

Ability to automate infrastructure provisioning and configuration Management 

3 

Collaboration and Communication 

Mandatory 

Excellent communication and collaboration skills with stakeholders 

3