Search by job, company or skills

TVH Consulting

Data Platform Engineer

Save
  • Posted a day ago
  • Be among the first 10 applicants
Early Applicant

Job Description

As a Data Platform Engineer, you will be a key contributor in building and maintaining the new Operational Data Store. You will develop the infrastructure that captures, stores, and publishes master data across the enterprise, transitioning away from legacy point-to-point integrations to a highly decoupled event-driven architecture

Expanded Responsibilities

  • Pipeline Engineering: Build and maintain batch and real-time data pipelines using Dataflow and Kafka.
  • Orchestration: Develop and monitor complex Airflow (Cloud Composer) DAGs to ensure data arrives on time.
  • Infrastructure: Use Terraform to deploy and manage GCP resources, ensuring environmental consistency.
  • Software Development: Create internal APIs and microservices using FastAPI and Golang to enhance platform utility.
  • Data Modelling: Use dbt to transform raw data into clean, tested models within BigQuery.
  • Reliability: Manage Docker containers and CI/CD pipelines (GitLab/Jenkins) to support seamless deployments.

Technical Profile Additions

  • Proficiency: Python (Advanced), Golang (Intermediate), and SQL.
  • GCP Expertise:
    • Cloud Run (Services, Jobs, Functions)
    • Databases: BigQuery, Cloud SQL for PostgreSQL, Cloud Spanner
    • Cloud Storage
    • Managed Airflow (Cloud Composer)
    • Pub/Sub, Eventarc
    • Artifact Registry
    • Secret Manager
Nice To Have

  • Project Management & Documentation: Jira, Confluence, ServiceNow
  • Frontend Technologies: React.js

Required Education

Educational Background & Qualifications

  • Master's or Bachelor's Degree in Computer Science, Data Engineering, Software Engineering, or a related technical field (e.g., Electromechanical Engineering with a focus on Automation)
  • Specialized Coursework: Strong academic foundation in Distributed Systems, Database Management Systems (DBMS), Algorithm Design, and Real-time Computing.

Preferred Certifications

  • Google Cloud Professional Data Engineer.

More Info

Job Type:
Industry:
Employment Type:

About Company

Job ID: 149016185

Similar Jobs

Pune, India

Skills:

react.js GolangKafkaApache AirflowCloud StorageDockerTerraformGitlabPythonBigQueryJenkinsIamFastAPIDataFlowCloud SpannerCloud ComposerVertex AIFivetranCloud SQL for PostgreSQLManaged AirflowGCP Monitoring and AlertingArtifact RegistryEventarcGCP NetworkingSecret Manager

Pune, India

Skills:

react.js GolangKafkaApache AirflowCloud StorageDockerTerraformGitlabPythonBigQueryJenkinsIamFastAPIDataFlowCloud SpannerPrefectCloud ComposerVertex AIFivetranCloud SQL for PostgreSQLManaged AirflowGCP Monitoring and AlertingEventarcArtifact RegistryGCP NetworkingSecret Manager

Pune, India

Skills:

GolangBigQueryKafkaSqlJenkinsCloud StorageGcpDockerTerraformGitlabFastAPIDataFlowPythonCloud SpannerAirflowManaged AirflowCloud ComposerCloud RundbtArtifact RegistryEventarcSecret ManagerCloud SQL for PostgreSQL

Pune, India

Skills:

PysparkPerformance TuningScalaSqlSparkSnowflake–Databricks integrationMonitoringproduction workloadsRcaDelta LakeTroubleshootingdata optimization techniquesDatabricks cluster management

Pune, India

Skills:

OpenshiftBashJsonRest ApisPythonKubernetesLinux fundamentalsSQL Postgres basics