Search by job, company or skills

Data Engineer

This job is no longer accepting applications

Job Description

Job Title: Data Engineer – GCP + PySpark + Scala

Experience: 6+ Years

We are looking for a skilled Data Engineer with hands-on experience in GCP, PySpark, and Scala to develop and optimize scalable batch ETL pipelines.

Key Skills

  • Strong experience in Scala, PySpark, GCP, BigQuery, Dataproc, and Apache Airflow
  • Experience in developing, deploying, monitoring, and optimizing batch ETL workflows
  • Good understanding of Medallion Architecture with data quality checks and data summarization
  • Expertise in performance optimization, observability, and runbook creation
  • Strong knowledge of Airflow orchestration, dependency handling, and idempotent execution
  • Exposure to Data Governance is an added advantage

Responsibilities

  • Design, develop, and maintain scalable ETL pipelines on GCP
  • Build and optimize data processing workflows using PySpark and Scala
  • Ensure data quality, performance, and reliability across data pipelines
  • Collaborate with cross-functional teams to deliver efficient data solutions

Skills: pyspark,pipelines,gcp,scala,etl

More Info

Job Type:
Industry:
Employment Type:

About Company

Job ID: 151197299

Similar Jobs

Chennai, India

Skills:

PysparkGcpScalaKafkaDataprocBQAirflow

Chennai, India

Skills:

PysparkAzure DatabricksSqlEncryptionAzure Data FactoryTerraformAzure Synapse AnalyticsData GovernanceArmPythonAzure DevOpsAzure PurviewADLS Gen2ComplianceSecurityrbacCI CD pipelinesDelta LakeBicepInfrastructure-as-Code

Chennai, India

Skills:

BigQueryDataFlowAdvanced SqlPythonLoggingDataformBatch and streaming data processingData CatalogCloud MonitoringData lakehouse architecture conceptsPub SubDataplexIAM Security controlsJava or ScalaData ingestion frameworksAnalytics HubData modeling and warehousing principlesCDC and incremental processing

Chennai, India

Skills:

BigQueryData ModelingCiKafkaSqlELTApache AirflowGitGcpDockerPythonKubernetesEtlcdGoogle Cloud Composer

Chennai, India

Skills:

distributed file system DebuggingOLAPTableauData LakeYarnSql ToolsVersion ControlGcpSparkCloud ServicesVerticaAWSEtlDremioData AnalysisTroubleshootingRelational DatabasesApache SupersetAgile data structuresMonitoring

Beware of Scammers

We don’t charge money for job offers