Data Lake / ETL Engineer - 2+ Years
Role Overview
We are seeking a Data Lake / ETL Engineer with 2+ years of experience in designing, developing, and maintaining data pipelines and ETL/ELT processes. The role focuses on building and optimizing data ingestion, transformation, and storage solutions that enable business analytics, AI/ML use cases, and secure enterprise data lakes.
Key Responsibilities- Pipeline Development
- Build and maintain ETL/ELT pipelines for structured and semi-structured data.
- Support data ingestion from databases, APIs, streaming platforms, and flat files.
- Ensure data quality, integrity, and lineage across data flows.
- Data Lake Engineering
- Assist in the design and development of data lake solutions on cloud and on-prem.
- Implement storage and retrieval mechanisms optimized for performance.
- Manage metadata and cataloging for discoverability and governance.
- Performance & Optimization
- Tune ETL workflows for efficiency and cost-effectiveness.
- Implement partitioning, indexing, and caching for large-scale data processing.
- Automate repetitive data preparation tasks.
- Collaboration & Support
- Work with data scientists and analysts to deliver clean and reliable datasets.
- Collaborate with senior engineers on best practices for data modeling and pipeline design.
- Provide L2 support for production pipelines and help troubleshoot failures.
Required Skills & Experience
- 2+ years of experience in data engineering or ETL development.
- Proficiency in SQL and Python (or Scala/Java) for data transformations.
- Hands-on with ETL tools (Informatica, Talend, dbt, SSIS, Glue, or similar).
- Exposure to big data technologies (Hadoop, Spark, Hive, Delta Lake).
- Familiarity with cloud data platforms (AWS Glue/Redshift, Azure Data Factory/Synapse, GCP Dataflow/BigQuery).
- Understanding of workflow orchestration (Airflow, Oozie, Prefect, or Temporal).
Preferred Knowledge
- Experience with real-time data pipelines using Kafka, Kinesis, or Pub/Sub.
- Basic understanding of data warehousing and dimensional modeling.
- Exposure to containerization and CI/CD pipelines for data engineering.
- Knowledge of data security practices (masking, encryption, RBAC).
Education & Certifications- Bachelor's degree in Computer Science, IT, or related field.
- Preferred certifications:
- AWS Data Analytics - Specialty / Azure Data Engineer Associate / GCP Data Engineer.
- dbt or Informatica/Talend certifications.
