Big Data Developer

Capco

5-10 Years

Not Disclosed

Pune, India

This job is no longer accepting applications

Job Description

Big Data Engineer – GCP & Scala

Location: Bengaluru and Pune

Experience: 5–10 Years

Apply here: https://forms.cloud.microsoft/r/Vngu7esJju

[Confidential Information]

We are looking for an experienced Data Engineer with strong expertise in Google Cloud Platform (GCP), Scala, and Apache Spark to design, develop, and maintain scalable data engineering solutions. The ideal candidate will have hands-on experience building cloud-native data pipelines, processing large-scale datasets, and implementing modern data architectures using GCP services and big data technologies.

Key Responsibilities

Design, develop, and optimize scalable batch and real-time data pipelines using Scala and Apache Spark.
Build and maintain ETL/ELT workflows for large-scale data processing.
Develop cloud-native data solutions on Google Cloud Platform (GCP).
Work with Hadoop ecosystem technologies, including Hadoop, Hive, and HDFS.
Develop and optimize SQL queries and database solutions using PostgreSQL and other relational databases.
Implement workflow orchestration using Apache Airflow/Cloud Composer.
Build CI/CD pipelines using Jenkins and follow DevOps best practices.
Apply cloud design patterns and architectural best practices to deliver secure, scalable, and high-performing solutions.
Collaborate with cross-functional teams including Data Architects, Business Analysts, Product Owners, and DevOps engineers.
Monitor, troubleshoot, and optimize production data pipelines and cloud resources.
Ensure data quality, governance, security, and performance standards are met.