GCP Data Engineer

Job Summary

We are seeking a skilled GCP Data Engineer with 5+ years of experience in designing, developing, and maintaining scalable data pipelines and cloud-based data solutions on Google Cloud Platform (GCP). The ideal candidate should have strong expertise in BigQuery, GCS, Pub/Sub, Cloud Composer (Airflow), DBT, data quality frameworks, and monitoring solutions.

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT data pipelines on GCP.
• Build and optimize data ingestion frameworks using Pub/Sub, GCS, and other GCP services.
• Develop and manage workflows using Cloud Composer (Apache Airflow).
• Create and optimize data models and transformations using DBT.
• Implement robust data quality checks, validation rules, and data quality scans.
• Design and maintain enterprise-scale data warehouses in BigQuery.
• Monitor production pipelines and implement proactive monitoring and alerting mechanisms.
• Troubleshoot and resolve data pipeline, performance, and quality issues.
• Collaborate with data architects, analysts, and business stakeholders to deliver data solutions.
• Ensure adherence to security, governance, and best practices across data platforms.
• Optimize query performance and cloud infrastructure costs.

Mandatory Skills

Google Cloud Platform (GCP)

• Google Cloud Storage (GCS)
• BigQuery
• Pub/Sub
• Cloud Composer (Apache Airflow)
• Monitoring & Alerting

Data Engineering

• ETL/ELT Development
• Data Pipeline Design and Development
• Data Warehousing Concepts
• Advanced SQL

Data Quality & Governance

• Data Quality Scans
• Data Validation Frameworks
• Data Monitoring and Observability

Data Transformation

• DBT (Data Build Tool)
• Data Modeling
• Performance Optimization

Programming

• Python
• SQL

Required Experience

• 5+ years of experience in Data Engineering.
• Minimum 3+ years of hands-on experience in GCP Data Engineering solutions.
• Strong experience with BigQuery, GCS, Pub/Sub, Cloud Composer (Airflow), and DBT.
• Experience in implementing monitoring, alerting, and data quality frameworks.
• Hands-on experience in designing and supporting production-grade data pipelines.

Preferred Skills

• Dataflow
• Terraform
• CI/CD Pipelines
• GitHub/GitLab
• Spark/PySpark
• Kubernetes

Educational Qualification

• Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field.