GCP Data Engineer

Responsibilities :

Responsibilities and JD in brief along with additional criteria to be considered:

Design and develop scalable GCP Data Engineering solutions and cloud-native data platforms.

Build batch and real-time pipelines using BigQuery, Cloud Storage, Dataflow, Pub/Sub, Dataproc, Python, PySpark and Databricks.

Develop and optimize Databricks, Delta Lake, Spark and Structured Streaming workloads for large-scale processing.

Implement high-throughput streaming solutions using Pub/Sub and Dataflow, including windowing, watermarking, late-arriving data, duplicate handling and dead-letter queues.

Handle high-volume TB/PB-scale data with focus on performance, scalability, reliability and cost optimization.

Develop complex SQL transformations, data models and analytics-ready BigQuery datasets using partitioning and clustering.

Orchestrate pipelines using Cloud Composer / Airflow and implement CI/CD and Terraform-based Infrastructure as Code.

Ensure data quality, governance, security, monitoring, alerting and production support.

Collaborate with architects, stakeholders and cross-functional teams in an Agile delivery model.

Troubleshoot and optimize end-to-end GCP, Databricks and streaming data pipelines.

Additional Responsibilities:

Infosys is a global leader in next-generation digital services and consulting. Within the Data & Analytics unit, you will work on cutting-edge data engineering and analytics initiatives for global clients across industries. The role offers opportunities to work with cloud-native data platforms, modern analytics ecosystems, AI-driven solutions, and large-scale enterprise data modernization programs. Employees benefit from continuous learning programs, certifications, and career growth opportunities within Infosys data and analytics practice

Technical and Professional Requirements:

1. GCP BigQuery, Cloud Storage, Dataflow, Pub/Sub, Dataproc, batch and real-time streaming, Databricks , Pyspark 2. Python, strong SQL Databricks with Delta Lake and Structured Streaming Responsibilities and JD in brief along with additional criteria to be considered:

Design and develop scalable GCP Data Engineering solutions and cloud-native data platforms.

Build batch and real-time pipelines using BigQuery, Cloud Storage, Dataflow, Pub/Sub, Dataproc, Python, PySpark and Databricks.

Develop and optimize Databricks, Delta Lake, Spark and Structured Streaming workloads for large-scale processing.

Implement high-throughput streaming solutions using Pub/Sub and Dataflow, including windowing, watermarking, late-arriving data, duplicate handling and dead-letter queues.

Handle high-volume TB/PB-scale data with focus on performance, scalability, reliability and cost optimization.

Develop complex SQL transformations, data models and analytics-ready BigQuery datasets using partitioning and clustering.

Orchestrate pipelines using Cloud Composer / Airflow and implement CI/CD and Terraform-based Infrastructure as Code.

Ensure data quality, governance, security, monitoring, alerting and production support.

Collaborate with architects, stakeholders and cross-functional teams in an Agile delivery model.

Troubleshoot and optimize end-to-end GCP, Databricks and streaming data pipelines.