Data Lake / ETL Engineer - 2+ Years

Role Overview

We are seeking a Data Lake / ETL Engineer with 2+ years of experience in designing, developing, and maintaining data pipelines and ETL/ELT processes. The role focuses on building and optimizing data ingestion, transformation, and storage solutions that enable business analytics, AI/ML use cases, and secure enterprise data lakes.

Key Responsibilities

Pipeline Development

Build and maintain ETL/ELT pipelines for structured and semi-structured data.
Support data ingestion from databases, APIs, streaming platforms, and flat files.
Ensure data quality, integrity, and lineage across data flows.

Data Lake Engineering

Assist in the design and development of data lake solutions on cloud and on-prem.
Implement storage and retrieval mechanisms optimized for performance.
Manage metadata and cataloging for discoverability and governance.

Performance & Optimization

Tune ETL workflows for efficiency and cost-effectiveness.
Implement partitioning, indexing, and caching for large-scale data processing.
Automate repetitive data preparation tasks.

Collaboration & Support

Work with data scientists and analysts to deliver clean and reliable datasets.
Collaborate with senior engineers on best practices for data modeling and pipeline design.
Provide L2 support for production pipelines and help troubleshoot failures.

Required Skills & Experience

2+ years of experience in data engineering or ETL development.
Proficiency in SQL and Python (or Scala/Java) for data transformations.
Hands-on with ETL tools (Informatica, Talend, dbt, SSIS, Glue, or similar).
Exposure to big data technologies (Hadoop, Spark, Hive, Delta Lake).
Familiarity with cloud data platforms (AWS Glue/Redshift, Azure Data Factory/Synapse, GCP Dataflow/BigQuery).
Understanding of workflow orchestration (Airflow, Oozie, Prefect, or Temporal).

Preferred Knowledge

Experience with real-time data pipelines using Kafka, Kinesis, or Pub/Sub.
Basic understanding of data warehousing and dimensional modeling.
Exposure to containerization and CI/CD pipelines for data engineering.
Knowledge of data security practices (masking, encryption, RBAC).

Education & Certifications

Bachelor's degree in Computer Science, IT, or related field.
Preferred certifications:

AWS Data Analytics - Specialty / Azure Data Engineer Associate / GCP Data Engineer.
dbt or Informatica/Talend certifications.