Data Engineer (GCP)

OB DESCRIPTION

Data Engineer – Job Description

We are looking for a Data Engineer to build and manage data pipelines on GCP for:

• Real-time data ingestion (CDC pipelines)

• Data transformation and modeling in BigQuery

• Supporting AI use cases (NL → SQL, Voice Bot)

The role involves developing scalable ETL/ELT pipelines, working with streaming and batch data, and ensuring data is reliable,

optimized, and ready for analytics and AI consumption.

Experience Required:

• Total Experience: 4–6 years

• Relevant Experience: 2–3 years on GCP / modern data platforms

Tools & Projects the Candidate Will Work On

Tools / Technologies

• BigQuery (data storage, transformations)

• GCS (data ingestion layer)

• Dataflow / Pub-Sub (streaming pipelines)

• CDC Tools (Debezium / Kafka or similar)

• SQL + Python (data processing)

• Source system: Amazon Redshift

Projects

• Build real-time CDC pipelines (Redshift → BigQuery)

• Develop ETL/ELT pipelines for data ingestion and transformation

• Create curated datasets/views for AI use cases

• Support data migration and optimization in BigQuery

Ensure data quality, validation, and performance tuning

EXPERTISE AND QUALIFICATIONS

Skills & Expertise

• Strong experience in data pipeline development (batch + streaming)

• Hands-on with BigQuery and SQL-based transformations

• Experience with CDC and real-time data ingestion

• Ability to handle large-scale data processing

• Understanding of data modeling and optimization

Must-Have Skills

• Hands-on experience with BigQuery

• Strong SQL skills (joins, aggregations, performance tuning)

• Experience building ETL/ELT pipelines

• Exposure to real-time data pipelines / CDC

• Experience with GCP services (GCS, Dataflow, Pub/Sub)

• Good problem-solving and debugging skills

Good-to-Have Skills

• Experience with Debezium / Kafka

• Exposure to Redshift or similar data warehouse

• Knowledge of partitioning, clustering, and cost optimization

• Basic understanding of AI/ML data requirements

Experience in data validation and monitoring frameworks