Senior Data Engineer

We're Hiring: Senior Data Engineer

Location: Bangalore

Experience: 6+ Years

About the Role

We're looking for a Senior Data Engineer with strong data architecture and system design skills to design, build, and own end-to-end production pipelines powering analytics and decision-making across the organization. The role centers on deep Python and SQL expertise, robust transaction-level fact tables, and rigorous data quality and auditability, ideally within a finance, risk, or compliance data domain.

What You'll Do

Own data architecture and system design decisions for pipelines and data models, including grain, partitioning, schema evolution, and scalability trade-offs.

Take end-to-end ownership of production pipelines: design, build, deploy, monitor, and operate.

Design and build transaction tables (append-only, immutable event/transaction-grain fact tables) alongside standard fact/dimension models using medallion (bronze/silver/gold) architecture.

Build and support pipelines for finance, risk, or compliance use cases where accuracy, auditability, and data lineage are critical.

Implement data quality and auditability controls, including validation checks, reconciliation logic, anomaly detection, and audit trails for every pipeline.

Write complex, cross-dialect SQL (MySQL + PostgreSQL), including window functions, multi-layered CTEs, CASE-driven logic, COALESCE/NULLIF, type casting, and date/timestamp handling.

Build idempotent reload patterns (DELETE + INSERT), UNION ALL/set operations, and templated (Jinja-style) SQL, including handling late-arriving/corrected transaction records without double-counting.

Orchestrate Airflow-style DAGs with cross-pipeline dependencies and secure credential handling.

Build Python ETL tooling using pandas (CSV ↔ DB), SQLAlchemy + raw drivers (MySQLdb, psycopg2), type hints/dataclasses, and class-based pipeline design.

Integrate with Google Drive/Sheets APIs, build YAML-driven pipeline configurations, and handle pipeline staging requirements.

Own CI/CD for pipelines through standard Git workflows.

Document data models and pipeline logic for maintainability, keeping compliance and audit requirements in mind.

What We're Looking For

Must-Have Skills:

6+ years of experience in data engineering.

Strong data architecture and system design skills to make sound trade-offs on modeling, partitioning, and scalability—not just write queries.

Demonstrated end-to-end production ownership of data pipelines from design through deployment, monitoring, and operational support.

Expert-level Python and SQL as the primary toolset. Hive/Hadoop is a nice-to-have only.

Robust data quality and auditability practices, including validation, reconciliation, anomaly detection, and audit trails.

Experience working with finance, risk, or compliance data, with an understanding of accuracy, auditability, and data-lineage requirements.

Proven experience designing and building transaction/fact tables.

Expert SQL skills, including window functions, CTEs, CASE logic, and cross-dialect MySQL/PostgreSQL.

Strong Python skills with pandas, SQLAlchemy, type hints/dataclasses, and class-based pipelines.

Airflow-like orchestration experience.

Solid data modeling fundamentals, including fact/dimension models, normalization, and medallion architecture.

YAML-based configuration experience, along with standard CI/CD and Git workflows.

Nice-to-Have Skills

Hive/Hadoop or Spark experience.

Interested candidates: Share your updated resume

#Hiring #SeniorDataEngineer #DataEngineering #Python #SQL #DataArchitecture #ETL #DataPipelines #BangaloreJobs #TechHiring