Search by job, company or skills

Fresher
Not Disclosed
Early Applicant
  • Posted 15 hours ago
  • Be among the first 10 applicants

Job Description

Job Description

Skillset: (Senior Data Engineer)

  • . Hands-on experience with GCP (BigQuery, Composer, Dataproc, Dataflow, Pub/Sub)
  • . Working knowledge for data processing and pipelines
  • . Strong programming skills in SQL and PySpark
  • . Experience with Oracle and SQL Server databases
  • . Understanding of data modelling (dimensional / medallion basics)
  • . Experience in batch pipelines and exposure to streaming
  • . Familiarity with data ingestion, ETL/ELT processes
  • . Basic understanding of data quality and validation
  • . Exposure to Airflow/Composer and scheduling
  • . Working knowledge of Git and CI/CD

Detailed Responsibilities:

  • . Develop and maintain batch and streaming data pipelines
  • . Build SQL and PySpark transformations for large datasets
  • . Ingest data from multiple sources into BigQuery/cloud platforms
  • . Implement data transformation logic as per requirements
  • . Work with teams to consume curated datasets
  • . Develop workflows using Cloud Composer (Airflow)
  • . Perform data validation and reconciliation
  • . Optimise queries and pipelines for performance
  • . Support migration of legacy ETL jobs to cloud
  • . Troubleshoot pipeline and performance issues
  • . Participate in testing, UAT, and deployment
  • . Document pipelines, transformations, and sources
  • . Collaborate with analytics and reporting teams.

More Info

Job Type:
Employment Type:

Similar Jobs

India
Skills:
composer , BigQuery, Data Modelling, Pyspark, SQL Server, Dataproc, Sql, ELT, Data Quality, Git, Streaming, Gcp, DataFlow, Oracle, Etl, Airflow, CI CD, Pub Sub, Data Validation, batch pipelines
Chennai, India
Skills:
Apis, Pyspark, Python, LangChain, CrewAI, tool function calling, vector databases, AG2, Azure OpenAI, Arize, Azure AI Foundry, LangSmith, Semantic Kernel, LangGraph, AutoGen, OpenTelemetry, Langfuse, CI/CD, context management techniques, async code, LLM observability tooling
Pune, India
Skills:
.NET, C#, SQL Server, Microservices, Git, Docker, Agile, Entity Framework, Restful Apis, Azure, Web Apis, Kubernetes, Xunit, Azure DevOps, Azure Service Bus, ASP.NET Core
Pune, India
Skills:
data engineering , Leadership, Architecture, Data Quality Controls, Stakeholder Management
Chennai, India
Skills:
.NET, .Net Core, .NET MVC, C#, Devexpress, VB.NET, Ssrs, Restful Api, Json, Powershell Scripting, Javascript, Python, Java, Sql, Redis, Rabbitmq, jQuery, Xml, ASP.NET, Etl, REST-based web services, Power Apps, Angular 20, ASP.NET Web Forms, CI/CD