Data Engineer

Project Role : Data Engineer

Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems.

Must have skills : Data Build Tool

Good to have skills : NA

Minimum 5 Year(s) Of Experience Is Required

Educational Qualification : 15 years full time education

Data Engineer (Contract)

Duration: 6 months (with potential to expand capacity) Engagement: Starting with 1 resource

Role Overview

We are seeking an experienced Data Engineer to build and maintain robust, scalable data pipelines on our cloud data platform.

You will work within a modern Medallion architecture, delivering high-quality, well-tested data models and pipelines using dbt, Python, and modern orchestration tools.

Key Responsibilities

Design, build, and maintain dbt data models across Bronze, Silver, and Gold layers of a Medallion architecture

Develop custom dbt macros to enforce modeling standards, reduce code duplication, and improve team productivity

Write production-grade Python code for data ingestion, transformation, and processing workflows

Build, schedule, and monitor data pipelines using orchestration platforms such as Dagster and Astronomer (Apache Airflow)

Implement and manage CI/CD pipelines in GitLab for automated testing, linting, and deployment of dbt projects and Python packages

Collaborate with data architects and analytics engineers to translate business requirements into clean, performant data models and pipelines

Apply data modeling best practices including normalization, slowly changing dimensions, and star/snowflake schema design where appropriate

Leverage AI-assisted development tools such as Amazon Q and Kiro to accelerate pipeline development

Required Skills

Expert-level proficiency in dbt (Core or Cloud), including advanced features such as custom macros, hooks, tests, and packages

Strong Python development skills, including writing modular, testable, and maintainable code for data engineering use cases

Experience building and managing orchestrated workflows using Dagster and/or Astronomer (Apache Airflow), including DAG authoring, sensors, and dependency management

Strong experience with CI/CD pipelines using GitLab, including .gitlab-ci.yml authoring and pipeline optimization

Solid understanding of data modeling principles and experience implementing Medallion (Bronze/Silver/Gold) architecture

Proficiency in SQL and experience working with cloud data warehouses (Snowflake preferred)

Exposure to AWS data services (S3, Glue, Lambda) in a data pipeline context

Familiarity with data quality frameworks such as Great Expectations or dbt tests

Familiarity with AI-assisted development tools (Amazon Q, Kiro, or similar)

Nice to Have (will be mandatory skill set for future adds)

Experience with PySpark for distributed data processing at scale