Senior Data Engineer

Job Title : Senior Data Engineer

Job Location : Bengaluru

Exp : 6-10 years

Notice Period : 15 days

Design and operate large-scale Spark batch pipelines on HDFS/Hive, ensuring correctness under late-arriving data, reprocessing, and overwrite-only constraints.

Own data modeling, deduplication, aggregation, and reconciliation logic to deliver deterministic, auditable datasets at scale.

Implement idempotent processing patterns, backfills, and partition-level validation to guarantee reproducibility and trust.

Build observable and reliable data systems with strong data quality checks, lineage, SLAs, and operational metrics.

Optimize performance and cost through efficient partitioning, Spark tuning, and storage/query design.

Partner with analytics, data science, and platform teams to translate business requirements into scalable data assets.

Act as a technical owner for data reliability, driving standards, best practices, and mentoring across the data platform.

Must haves

Experience designing large-scale data models for analytics and reporting.

Proficiency in SQL and at least one programming language such as Scala or Python.

Familiarity with workflow orchestration (e.g., Airflow, Oozie, or similar).

Experience with data quality, validation, and reconciliation frameworks.

Knowledge of distributed systems concepts (partitioning, fault tolerance, consistency).