Search by job, company or skills

Fresher
Not Disclosed
Early Applicant
  • Posted 15 hours ago
  • Be among the first 10 applicants

Job Description

Job Description

Skillset: (Lead Data Engineer)

  • . Hands-on experience with GCP (BigQuery, Composer, Dataproc, Dataflow, Pub/Sub)
  • . Working knowledge for data processing and pipelines
  • . Strong programming skills in SQL and PySpark
  • . Experience with Oracle and SQL Server databases
  • . Understanding of data modelling (dimensional / medallion basics)
  • . Experience in batch pipelines and exposure to streaming
  • . Familiarity with data ingestion, ETL/ELT processes
  • . Basic understanding of data quality and validation
  • . Exposure to Airflow/Composer and scheduling
  • . Working knowledge of Git and CI/CD

Detailed Responsibilities:

  • . Develop and maintain batch and streaming data pipelines
  • . Build SQL and PySpark transformations for large datasets
  • . Ingest data from multiple sources into BigQuery/cloud platforms
  • . Implement data transformation logic as per requirements
  • . Work with teams to consume curated datasets
  • . Develop workflows using Cloud Composer (Airflow)
  • . Perform data validation and reconciliation
  • . Optimise queries and pipelines for performance
  • . Support migration of legacy ETL jobs to cloud
  • . Troubleshoot pipeline and performance issues
  • . Participate in testing, UAT, and deployment
  • . Document pipelines, transformations, and sources
  • . Collaborate with analytics and reporting teams.

More Info

Job Type:
Employment Type:

Similar Jobs

India
Skills:
Apis, Reporting Tools, Sql, Labor forecasting, Healthcare-specific workforce rules, Payroll integrations, UKG Pro WFM, Kronos WFC Dimensions, Timekeeping policies
Pune, India
Skills:
.NET, C#, SQL Server, Microservices, Git, Docker, Agile, Entity Framework, Restful Apis, Azure, Web Apis, Kubernetes, Xunit, Azure DevOps, Azure Service Bus, ASP.NET Core
Pune, India
Skills:
Data Lineage, Servicenow, Pyspark, Sql, Python, Pipeline Builder, Palantir Foundry, Data Connection, Code Repositories
Mumbai, India
Skills:
Version Control, Backend Development, Sql, Relational data modelling, Testing discipline, Data quality and reconciliation, Communication in a distributed team, Pipeline construction and operation, Python for data engineering
1-3 yrs
Bengaluru, India
Skills:
Java, Jms, Unix, Hibernate, Linux, MySQL, J2EE, Json, Oracle, Spring