GCP Data Architect

Role- GCP Data Architect

Experience-12–18 years overall experience; 4–8+ years GCP; multiple platform implementations

Location- PAN INDIA

Required Skills & Expertise

• GCP Services: BigQuery, GCS, Pub/Sub, Dataflow, Dataproc, Dataplex, Dataform, Composer
• Programming: Python, PySpark, Beam, SQL
• Streaming: Kafka, Pub/Sub
• IaC & DataOps: Terraform, Source Repository, Github, Cloud Build, Jenkins, Artifact Registry
• Data Governance: IAM, DLP, tagging, lineage tools
• Data Migration: Data Migration Services, Dataproc, Dataflow, Data Fusion, Data Stream, Storage Transfer

Role Summary

Seasoned GCP Data Architect with expertise in design, implement, and operationalizing enterprise-grade data analytics platforms on Google Cloud. Strong experience in platform engineering, data migration strategies, data ingestion patterns, governance, observability, DataOps, and scalable data architecture.

Core Responsibilities

1. GCP Data Analytics Platform (Greenfield & Modernization)

Design and deploy enterprise data platform architecture; environment setup; architecture pattern - data Lakehouse / data mesh architecture; data observability, Infrastructure Provisioning, VPC Setup. IAM Policies / Service Accounts, Data Protection at Rest and Transit, Guardrails.

2. Data Migration Strategy

Design and implement end-to-end data migration strategies from on-prem/legacy systems (Historical and Incremental) to GCP. (Data Migration Service, BigQuery Migration Service)

3. Data Ingestion Patterns

Design and implement scalable ingestion frameworks supporting multiple pattens - Batch / Realtime / API-Based Ingestions (GSUtil, Kafka, Pub Sub, Data Stream)

4. Data Engineering Pipelines (Batch and Streaming)

Build and optimize the end-to-end data engineering batch and Realtime / near Realtime steaming pipelines to build data products / data marts (BigQuery, Dataflow, Dataproc, Dataform, DBT, GCS, Composer)

5. Data Governance & Security

Implement enterprise grade data governance and compliances – data security RBAC controls, data classification and tagging, data discovery, data lineage, metadata management. (Dataplex, IAM, DLP, Data Catalog)

6. Data Observability Framework

Implement centralized monitoring frameworks - data engineering pipeline monitoring, data lineage tracking and alerting mechanism (Cloud Logging and Monitoring)

7. DataOps & Platform Automation

Implement data operation / orchestration pipeline to automate build, test and deployments with infrastructure provisioning automation (Source Repository, Cloud Build, Artifact Registry, Composer, Terraform)

6. Performance & Cost Optimization

Optimize platform performance and cost – BigQuery Optimization – Partitioning and Clustering, Query Optimization, Storage Life Cycle policies, FinOps (BigQuery, Denormalization, Star and Snowflake Schema, Query Tuning, FinOps)