Lead Data Engineer

This is a remote position.

Job Description – Lead Data Engineer
Experience: 7–10 Years
Location: Remote / Hybrid (Bengaluru)
Notice Period: Immediate Joiners Only

Domain:Data Platforms & Analytics Infrastructure

About the Role

We are looking for a highly skilled and experienced Lead Data Engineer to build and scale modern data platforms that power analytics, reporting, and AI/ML initiatives. In this role, you will own the end-to-end data lifecycle, architect scalable data solutions, and lead a team of engineers to deliver reliable, high-performance data infrastructure.

The ideal candidate should have deep expertise in data engineering, distributed processing, cloud platforms, and modern Lakehouse architectures, along with strong leadership and mentoring capabilities.

Key Responsibilities

• Design, build, and maintain scalable batch and real-time data pipelines.
  

• Architect and implement modern Lakehouse solutions using technologies such as Delta Lake or Apache Iceberg.
  

• Develop and manage ETL/ELT workflows using tools like Airflow, dbt, or Prefect.
  

• Implement robust data quality, governance, lineage, and monitoring frameworks.
  

• Design optimized data models to support analytics, business intelligence, and ML workloads.
  

• Improve platform scalability, reliability, performance, and cost efficiency.
  

• Collaborate with Analytics, Product, and AI/ML teams to enable data-driven solutions.
  

• Lead and mentor a team of data engineers while driving engineering best practices and standards.
  

• Build and maintain CI/CD pipelines and infrastructure automation for data platforms.
  

• Ensure platform reliability, observability, and operational excellence across the data ecosystem.
  

Required Skills & Qualifications

• 7–10 years of experience in Data Engineering, with at least 3+ years in a lead or mentoring role.
  

• Strong expertise in SQL, Python, and Spark (PySpark/Scala).
  

• Hands-on experience with modern cloud data platforms such as: Snowflake ,Databricks ,BigQuery ,Amazon Redshift
  
• Strong understanding of data modeling methodologies including Kimball and Data Vault.
  

• Experience working with streaming and real-time data systems such as Kafka, Flink, or similar technologies.
  

• Familiarity with infrastructure-as-code and DevOps practices using Terraform and CI/CD pipelines.
  

• Hands-on experience with cloud platforms including AWS, GCP, or Azure.
  

• Strong understanding of scalable distributed data systems and modern data architecture patterns.
  

Preferred Qualifications

• Experience with feature stores such as Feast or Tecton.
  

• Familiarity with data mesh concepts and CDC tools like Fivetran or Airbyte.
  

• Exposure to graph databases and vector databases.
  

• Experience contributing to open-source projects is a plus.
  

• Advanced degree in Computer Science, Data Engineering, or a related field preferred.
  

Tech Stack

• Programming: Python, SQL, PySpark, Scala
  

• Data Platforms: Snowflake, Databricks, BigQuery
  

• Workflow Orchestration: Airflow, dbt, Prefect
  

• Streaming: Kafka
  

• Cloud: AWS / GCP / Azure
  

• Infrastructure & DevOps: Docker, Kubernetes, Terraform