GCP Data Engineer

Role: GCP Data Engineer

Exp: 6+ Years

Loc: Hyderabad

Job Description

About this role:

Experienced with a strong background in Data Engineering to join our team. The ideal candidate will have expertise in Python, Apache Beam, and experience working with Google Cloud Platform (GCP) environments. Additionally, strong SQL and analytical skills are mandatory for this role. You will play a critical role in designing, developing, and optimizing scalable data pipelines and solutions to support our business needs.

Job Expectations:

• Contribute to design, development, and implementation of end-to-end data pipelines using Python and Apache Beam.
• Architect and optimize data workflows for GCP environments, ensuring scalability, reliability, and performance.
• Write complex SQL queries for data extraction, transformation, and analysis, ensuring data accuracy and consistency.
• Collaborate with cross-functional teams, including data scientists, analysts, and DevOps, to understand data requirements and deliver solutions.
• Provide technical contributions for a squad of engineers, ensuring best practices in coding, testing, and deployment.
• Develop and maintain data models, ETL processes, and data integration solutions.
• Perform data analysis to identify trends, anomalies, and insights to support business decision-making.
• Ensure data security, compliance, and governance across all environments.
• Monitor and troubleshoot data pipelines, addressing performance bottlenecks and failures.
• Stay up to date with the latest trends and technologies in data engineering, cloud computing, and distributed systems.

Desired Qualifications:

• Expertise in Python with Apache Beam for building data pipelines.
• Strong expertise in GCP environments, including services like BigQuery, Apache IceBerg, Dataflow, Pub/Sub, and Cloud Storage.
• Advanced SQL skills, with the ability to write and optimize complex queries for large datasets.
• Strong analytical skills, with experience in interpreting data and providing actionable insights.
• Solid understanding of distributed systems, data processing frameworks, and ETL/ELT workflows.
• Experience with CI/CD pipelines, containerization (Docker), and orchestration tools (Kubernetes, Airflow).
• Familiarity with relational and NoSQL databases, as well as data modeling concepts.
• Excellent problem-solving skills and the ability to work in a fast-paced, dynamic environment.

Additional Skills (Good to have):

• Experience with other data processing frameworks like Spark or Flink.
• Knowledge of data governance, security, and compliance best practices.
• Certification in GCP (e.g., Professional Data Engineer) is a plus.
• Experience with hybrid cloud solutions and migrating workloads from On-Premises to GCP.