Search by job, company or skills

Data Engineer

Data Engineer

RandomTrees
5-7 Years
Not Disclosed
Early Applicant
  • Posted 11 hours ago
  • Be among the first 10 applicants

Job Description

We are seeking an experienced Data Engineer with 5+ years of hands-on expertise across the modern data stack to design, build, and maintain high-performance ELT pipelines. In this role, you will be the core technical owner of our analytics foundation—leveraging Snowflake, dbt, hosted on cloud infrastructure (AWS, Azure, or GCP). You will partner closely with data analysts, analytics engineers, and business stakeholders to ensure our data models are scalable, reliable, and governed.

Key Responsibilities

  • Data Pipeline Architecture & ELT: Design, develop, and optimize robust, idempotent ELT data pipelines moving data from diverse operational sources into Snowflake.
  • Transformation & Modeling (dbt): Architect modular, production-grade dbt projects. Implement dimension and fact tables (Kimball/medallion architecture), write reusable Jinja macros, and establish automated testing (data freshness, uniqueness, referential integrity).
  • Workflow Orchestration (Airflow): Build, monitor, and maintain production Airflow DAGs with proper error handling, SLA alerts, retries, and dependency management.
  • Snowflak & Optimization: Configure and optimize virtual warehouses, storage, clustering keys, zero-copy cloning, and Time Travel. Enforce cost-governance practices and resource monitors.
  • Cloud Infrastructure & Ingestion: Integrate cloud storage services (e.g., AWS S3, Azure Blob/ADLS, or Google Cloud Storage) with Snowflake using Snowpipe, external stages, and event-driven architectures.
  • Data Quality & Governance: Establish schema migrations, data testing frameworks, and role-based access control (RBAC), object tagging, and data masking policies.
  • CI/CD & DataOps: Implement version control best practices, automated deployment pipelines (GitHub Actions, GitLab CI, or Bitbucket Pipelines), and environment management for data assets.

Qualifications & Requirements

Must-Have:

  • Experience: 5+ years of dedicated professional experience in data engineering, data warehousing, or business intelligence backend systems.
  • Snowflake: Deep expertise in Snowflake architecture, query profiling, warehouse right-sizing, Snowpipe, and security implementations (RBAC, row-level security).
  • dbt (Data Build Tool): Proven track record modeling complex datasets in dbt Core or dbt Cloud, using incremental models, snapshots, tests, documentation, and Jinja/SQL macros.
  • Any Major Cloud Platform: Solid working experience with at least one major cloud provider (AWS, Azure, or GCP), particularly with IAM roles, object storage, and serverless compute.
  • Programming & SQL: Mastery of advanced SQL (window functions, CTEs, query plan optimization) and intermediate-to-advanced proficiency in Python for data processing and automation.
  • Version Control & CI/CD: Hands-on experience with Git branching workflows and automated testing/deployment pipelines.

Nice-to-Have:

  • Experience with streaming technologies (Apache Kafka, AWS Kinesis) or Snowflake Streaming Snowpipe.
  • Exposure to Snowflake Dynamic Tables, Streams & Tasks, or Snowpark.
  • Relevant certifications: SnowPro Core / Advanced, dbt Certified Developer, or Associate/Professional Cloud Architect/Data Engineer (AWS/Azure/GCP).
  • Familiarity with data observability tools (e.g., Monte Carlo, Datafold, Elementary).

More Info

Job Type:
Industry:
Employment Type:

Key Skills

GitLab CI

GitHub Actions

Bitbucket Pipelines

About Company

Similar Jobs

6-8 yrs
Chennai, India
Skills:
S3, Github, Performance Tuning, Agile Development Methodologies, Pyspark, Emr, Redshift, Sql, ELT, Lambda, Git, Iam, Data Warehousing, Python, Etl, AWS Data Services, Glue, Optimization of Spark applications, Athena
5-7 yrs
Chennai, India
Skills:
Kafka, API design, Data Modeling, Microservices, ELT, Distributed Systems, Python, Java, BigQuery, Hadoop, Data Quality, Hive, MLops, Spark, Data Governance, Databricks, Fault Tolerance, Kubernetes, Etl, Airflow, GenAI, LLM tooling, event-driven patterns, asynchronous processing, cloud-native practices, observability, Big Data ecosystem, HDFS, Delta Lake, CI/CD
8-10 yrs
Chennai, India
Skills:
Adf, Scala, SQL Server, SSIS, Sql, Git, Ssas, Spark, Data Lake, Data Warehousing, Databricks, Data Governance, Python, Etl, Azure DevOps, Security Architecture, CI/CD Workflows, Relational Databases, Unity Catalog, OLAP Cubes, Delta Tables, R
6-10 yrs
Chennai, India
Skills:
ELT, Cloud Storage, Python, Ml, Google Cloud Platform, Sql, Devops, Git, Data Governance, Etl, Data Streams, Airflow, GenAI, Data engineering services, Google BigQuery, Data Flow, Cloud Composer, API integrations, Informatica CDC, data observability, Ai, data fusion, dbt, CI/CD
4-6 yrs
Chennai, India
Skills:
Git, Pyspark, Databricks, Azure, Sql, Python, AWS