Google Cloud Data Engineer
About the Role
We are seeking a skilled Data Engineer with hands-on experience in Google Cloud Platform (GCP), particularly focusing on Cloud Composer (Airflow), Dataform, Python, and SQL. The ideal candidate will design, build, and maintain scalable data pipelines, ensuring efficient data flow across our analytics and operational systems.
Key Responsibilities
- Design, develop, and orchestrate ETL/ELT pipelines using Cloud Composer (Airflow) for batch and near real-time data processing.
- Build and maintain modular and version-controlled data transformations using Dataform within BigQuery.
- Develop Python-based automation scripts, utilities, and data ingestion frameworks.
- Write and optimize complex SQL queries, stored procedures, and transformations to handle large-scale datasets in BigQuery.
- Collaborate with data analysts, data scientists, and business stakeholders to translate data requirements into technical specifications.
- Implement data quality checks, lineage tracking, and validation frameworks.
- Ensure pipeline reliability, observability, and performance tuning for production workloads.
Required Skills & Qualifications
- 5+ years of experience in data engineering or related fields.
- Strong programming skills in Python, including experience with libraries like pandas, sqlalchemy, or google-cloud-bigquery.
- Advanced proficiency in SQL with a deep understanding of analytical functions, query optimization, and schema design.
- Proven hands-on experience with Cloud Composer (Apache Airflow) DAG design, task dependencies, custom operators, sensors, and Airflow best practices.
- Experience building modular data pipelines using Dataform or similar frameworks (e.g., dbt).
- Experience working with BigQuery and GCP services such as Cloud Storage, Pub/Sub, Dataflow, and IAM.
- Understanding of data warehousing principles, data modeling, and ETL/ELT best practices.
