GCP DE

Job Description

• mandatory
  
  

SN

Required Information

Details

1

Role**

GCP Data Engineer

2

Required Technical Skill Set**

Data Engineer with hands-on experience in Google Cloud Platform

(GCP) services to work on implementing and managing data ingestion.

Work with services such as GCS, BigQuery, Dataflow, Composer, Data Fusion, DataProc for ETL/ELT operations

3

No of Requirements**

60+

4

Desired Experience Range**

5-12 Years

5

Location of Requirement

Pune, Hyderabad, Bangalore, Chennai, Gurgaon

Desired Competencies (Technical/Behavioral Competency)

Must-Have**

• 5+ years of experience as a Data Engineer or in a similar role.
• Strong working knowledge of Google Cloud Storage (GCS), BigQuery,
  
  

Composer, Data Fusion, Dataproc.

• Proficiency in Python for scripting, debugging, and automation tasks.
• Experience with Terraform for infrastructure-as-code.
• Knowledge of Git for version control.
• Understanding of data ingestion concepts, file formats, and ETL workflows.
• Ability to debug and resolve runtime data issues independently.
• Strong problem-solving and analytical skills.
  
  

Good-to-Have

• Exposure to TWSd or other enterprise job schedulers.
• Basic understanding of SQL optimization in BigQuery.
  
  

SN

Responsibility of / Expectations from the Role

• Configure and execute data ingestion pipelines using our reusable GCP-based
• framework.
• Work with services such as GCS, BigQuery, Dataflow, Composer, Data Fusion, DataProc for ETL/ELT operations.
• Strong experience in data engineering/migration projects in GCP (along with Python) is required and should be able to work independently.
• Manage parameters, job configurations, and metadata for ingestion.
• Debug and resolve issues related to data, parameters, and job execution.
• Escalate framework-related bugs to the core development team when required.
• Monitor daily job runs, troubleshoot failures, and ensure SLAs are met.
• Collaborate with cross-functional teams for smooth delivery.
• Follow version control best practices using Git.
• Maintain deployment scripts and infrastructure configurations via Terraform.
• (Nice-to-have) Work with TWSd job scheduling/monitoring tools.
• Suggest improvements to the framework or processes based on usage
• experience.
• Contribute to small enhancements in the framework, where applicable.
  
  

Job Description

• mandatory
  
  

SN

Required Information

Details

1

Role**

GCP Data Engineer

2

Required Technical Skill Set**

Data Engineer with hands-on experience in Google Cloud Platform

(GCP) services to work on implementing and managing data ingestion.

Work with services such as GCS, BigQuery, Dataflow, Composer, Data Fusion, DataProc for ETL/ELT operations

3

No of Requirements**

60+

4

Desired Experience Range**

5-12 Years

5

Location of Requirement

Pune, Hyderabad, Bangalore, Chennai, Gurgaon

Desired Competencies (Technical/Behavioral Competency)

Must-Have**

• 5+ years of experience as a Data Engineer or in a similar role.
• Strong working knowledge of Google Cloud Storage (GCS), BigQuery,
  
  

Composer, Data Fusion, Dataproc.

• Proficiency in Python for scripting, debugging, and automation tasks.
• Experience with Terraform for infrastructure-as-code.
• Knowledge of Git for version control.
• Understanding of data ingestion concepts, file formats, and ETL workflows.
• Ability to debug and resolve runtime data issues independently.
• Strong problem-solving and analytical skills.
  
  

Good-to-Have

• Exposure to TWSd or other enterprise job schedulers.
• Basic understanding of SQL optimization in BigQuery.
  
  

SN

Responsibility of / Expectations from the Role

• Configure and execute data ingestion pipelines using our reusable GCP-based
• framework.
• Work with services such as GCS, BigQuery, Dataflow, Composer, Data Fusion, DataProc for ETL/ELT operations.
• Strong experience in data engineering/migration projects in GCP (along with Python) is required and should be able to work independently.
• Manage parameters, job configurations, and metadata for ingestion.
• Debug and resolve issues related to data, parameters, and job execution.
• Escalate framework-related bugs to the core development team when required.
• Monitor daily job runs, troubleshoot failures, and ensure SLAs are met.
• Collaborate with cross-functional teams for smooth delivery.
• Follow version control best practices using Git.
• Maintain deployment scripts and infrastructure configurations via Terraform.
• (Nice-to-have) Work with TWSd job scheduling/monitoring tools.
• Suggest improvements to the framework or processes based on usage
• experience.
• Contribute to small enhancements in the framework, where applicable.
  
  

Skills: gcp,python,big query,cloud,composer