Lead Data Engineer

Lead Data Engineer | 9+ Years | Bangalore/Hyderabad

We're hiring a Lead Data Engineer to drive our enterprise Data Warehouse modernization to Google BigQuery & GCP!

Location: Bangalore / Hyderabad

Experience: 9+ years in Data Engineering/EDW, with 4+ years in a Lead/Principal role

Short joiners preferred

What you'll do:

Lead migration & modernization of legacy Oracle EDW/Exadata workloads to BigQuery and modern Lakehouse architectures

Design, build, and optimize large-scale batch & real-time ETL/ELT pipelines using Apache Spark (PySpark/Scala), Python, and Cloud Composer (Airflow)

Implement CDC and continuous replication from Oracle to BigQuery using Oracle GoldenGate, Kafka, and CDC frameworks

Establish end-to-end DataOps automation — CI/CD, data quality/reconciliation checks, and Infrastructure-as-Code (Terraform)

Champion AI/LLM-driven development — using tools like GitHub Copilot, Claude, ChatGPT, and Gemini for code generation, SQL refactoring, testing, and debugging

Set engineering standards, lead code/architecture reviews, and mentor teams on data engineering and AI-augmented workflows

What we're looking for:

9+ years in Data Engineering/Enterprise Data Warehousing, incl. 4+ years in a lead/principal capacity

Deep expertise in BigQuery — architecture, partitioning, clustering, slot optimization, cost governance & security (policy tags, row/column access policies)

Strong Oracle & SQL background — Exadata, advanced PL/SQL, complex data modeling (Star/Snowflake, Data Vault, SCDs), query tuning

Advanced hands-on experience with Apache Spark (PySpark/Scala) and Python for distributed data processing

Strong experience with Oracle GoldenGate, Kafka, or CDC streaming architectures

Proven track record automating data workflows and CI/CD pipelines (GitHub Actions, Jenkins, Terraform)

Demonstrated hands-on use of AI/LLMs in daily development — prompt engineering for code/SQL generation, automated testing, and code modernization