Senior Data Engineer

Role: Data Engineering, OpenShift Telemetry, Kafka, Splunk, LLM

Location: Pune

Experience: 8-15 Years

Mandatory Requirements

• Strong experience in Data Engineering using Python/Scala, ETL/ELT pipelines

• Hands‑on experience with OpenShift monitoring & telemetry stack (Prometheus, Grafana, OpenShift Metrics, Alerts)

• Deep knowledge of Kafka (producers, consumers, streaming pipelines, partitions, offsets)

• Strong experience in Splunk dashboards, SPL queries, alerting & log analysis

• Exposure to LLMs / Generative AI, prompt engineering, API integration

• Good understanding of container platforms, microservices, CI/CD

• Strong analytical and debugging skills

Optional / Good-to-Have Requirements

• Experience with DataOps & pipeline orchestration tools (Airflow, Argo)

• Knowledge of streaming frameworks (Kafka Streams / Spark Structured Streaming / Flink)

• Experience with cloud platforms (AWS/Azure/GCP)

• Exposure to MLOps & vector databases (Pinecone, Chroma, FAISS)

• Kubernetes admin basics

Job Description

• Build and manage scalable data pipelines and ingestion frameworks

• Develop and maintain Kafka‑based streaming pipelines for real‑time data processing

• Configure and monitor OpenShift telemetry, create metrics, and troubleshoot cluster issues

• Use Splunk for monitoring, log analysis, dashboarding, and alerting

• Integrate and operationalize LLM‑based solutions with enterprise systems

• Collaborate with platform, DevOps, and application teams for end‑to‑end data flow and reliability

• Implement quality, performance tuning, and automation across data workflows

• Participate in design reviews, capacity planning, and architecture discussions