Reflections Info Systems - Senior Data Engineer - Google Cloud Platform
Reflections Info Systems - Senior Data Engineer - Google Cloud Platform
Vista Info SystemsEarly Applicant
- Posted a month ago
- Be among the first 10 applicants
Job Description
Key Responsibilities
- Design and develop batch and real-time data pipelines on GCP.
- Build scalable ETL/ELT pipelines using Dataflow, BigQuery, and Cloud Composer.
- Ingest data from databases, APIs, files, SaaS applications, and streaming sources.
- Design and optimize BigQuery datasets, tables, partitioning, and clustering.
- Develop data transformations using SQL, Python, Apache Beam, Spark, dbt, or Dataform.
- Implement workflow orchestration and dependency management using Apache Airflow / Cloud Composer.
- Implement data quality checks, reconciliation, lineage, monitoring, and validation processes.
- Configure and manage GCP IAM, service accounts, and least-privilege access.
- Optimize BigQuery and Dataflow workloads for performance, scalability, and cloud cost efficiency.
- Build and manage infrastructure using Terraform.
- Participate in CI/CD implementation and deployment processes.
- Troubleshoot and resolve production data-pipeline failures and performance issues.
- Collaborate with Data Architects, Data Analysts, Data Scientists, and application engineering teams.
- Ensure data pipelines are reliable, secure, scalable, and maintainable.
- 6+ years of experience in Data Engineering.
- Strong hands-on experience with Google Cloud Platform (GCP).
- Expertise in BigQuery, Dataflow, and Cloud Composer/Airflow.
- Strong SQL and Python programming skills.
- Experience with Apache Beam / Spark.
- Experience with ETL/ELT pipeline development and data integration.
- Knowledge of dbt / Dataform.
- Strong understanding of data warehousing, data modelling, partitioning, and clustering.
- Experience with data quality, reconciliation, lineage, monitoring, and troubleshooting.
- Hands-on experience with GCP IAM and service accounts.
- Experience with Terraform and Infrastructure as Code (IaC).
- Understanding of CI/CD and DevOps practices.
- Strong experience designing production-grade, scalable data platforms on GCP.
- Experience handling both batch and streaming data workloads.
- Strong understanding of cloud cost optimization and performance tuning.
- Excellent analytical and problem-solving skills.
- Ability to work effectively with technical and business stakeholders.
- Strong communication and collaboration skills.
More Info
Key Skills
About Company