Role: Data Engineer II (SDE-2)

Experience: 3–5 Years

Joining: IMMEDIATE

About the Role

We are looking for a Data Engineer II (SDE-2) to join our Data Platform team and build scalable, high-performance data infrastructure powering real-time analytics and business-critical applications. In this role, you will design and develop modern Lakehouse architectures, build low-latency data pipelines, and optimize distributed data processing systems on AWS. You will work closely with Product, Engineering, and Analytics teams to deliver reliable, scalable, and efficient data solutions.

Key Responsibilities

Design, build, and maintain scalable batch and real-time data pipelines.
Develop and manage Change Data Capture (CDC) pipelines using technologies like Debezium, PeerDB, or similar.
Design and optimize Lakehousearchitectures using Apache Iceberg, Delta Lake, or Apache Hudi.
Build robust ETL/ELT pipelines using PySpark and Apache Flink.
Manage and optimize data infrastructure on AWS (S3, EMR, EKS, MSK).
Tune distributed query engines such as Trino or ClickHouse for high-performance analytics.
Design dimensional data models including Fact Tables, Dimension Tables, SCDs, and OBTs.
Automate deployments and infrastructure using GitLab/GitHub Actions and Terraform.
Drive technical design discussions, architecture reviews, and documentation.
Collaborate with Product and Business teams to deliver data solutions that enable analytics and decision-making.

Required Skills

3–5 years of experience in Data Engineering building scalable data platforms.
Strong programming skills in Python (PySpark) and SQL.
Hands-on experience with Apache Spark and distributed data processing.
Experience building batch and streaming data pipelines.
Strong knowledge of AWS services such as S3, EMR, EKS, and MSK.
Experience with workflow orchestration tools like Airflow or Temporal.
Understanding of modern Lakehouse technologies (Iceberg, Delta Lake, or Hudi).
Experience with distributed query engines such as Trino or ClickHouse.
Good understanding of data modeling concepts including Fact/Dimension modeling and Slowly Changing Dimensions (SCDs).
Strong problem-solving skills and ability to work in a fast-paced product engineering environment.

Good to Have

Experience with Kafka, Debezium, or PeerDB.
Exposure to Terraform, GitLab CI/CD, or GitHub Actions.
Programming experience in Go, Java, or Scala.
Experience using AI-assisted development tools such as GitHub Copilot, Claude, or Codex.

Tech Stack

Languages: Python, SQL (Go/Java/Scala – good to have)
Data Processing: PySpark, Apache Flink
Streaming & CDC: Kafka (MSK), Debezium, PeerDB
Storage: Apache Iceberg, DeltaLake, Apache Hudi
Cloud: AWS (S3, EMR, EKS, MSK)
Query Engines: Trino, ClickHouse
Orchestration: Airflow, Temporal
DevOps: GitLab CI/CD, GitHub Actions, Terraform
BI & Visualization: Metabase, Superset, Tableau, Power BI

AWS Data Engineer - SDE 2