Search by job, company or skills

Senior Data Engineer

Senior Data Engineer

Valuelabs
6-11 Years
Not Disclosed
Quick Apply
  • Posted a day ago
  • Over 50 applicants have applied

Job Description

Job Description

We are looking for Data Engineers with hands-on experience in building and managing scalable data pipelines using PySpark, Hive, SQL and Apache Oozie. The role involves developing reliable data solutions, optimizing performance and supporting production data platforms.

Roles & Responsibilities

  • Design, develop and maintain batch data pipelines using PySpark, Hive, SQL and Oozie.
  • Build reusable ETL/ELT workflows for data ingestion and transformation.
  • Optimize Spark jobs, SQL queries and data processing performance.
  • Implement data quality, validation, security and governance practices.
  • Manage workflow scheduling, monitoring and production troubleshooting.
  • Collaborate with architects, business teams and engineering stakeholders.
  • Maintain technical documentation and support CI/CD deployments.
  • Senior engineers will lead design reviews, code reviews and mentor team members.

Mandatory Skills

  • Python, PySpark, Apache Oozie
  • Apache Hive / HiveQL, SQL
  • Distributed Data Platforms
  • Data Engineering and ETL/ELT
  • Data Modeling and Data Warehousing
  • Spark Performance Tuning
  • Data Quality and Data Governance
  • Workflow Scheduling and Monitoring
  • Git and CI/CD

Good-to-Have Skills

  • Cloudera Data Platform (CDP)
  • Data Lakes and Data Lineage
  • Banking / Financial Services Domain
  • GitOps and Production Support
  • Claude Code / GitHub Copilot

Experience

  • Data Engineer: 3+ years
  • Senior Data Engineer: 6+ years

Strong analytical, problem-solving, communication and collaboration skills are essential.

More Info

Job Type:
Function:
Employment Type:

Key Skills

Distributed Data Platforms

ETL/ELT Development

About Company

Similar Jobs

5-7 yrs
Chennai, India
Skills:
snowflake , Cloudformation, Pyspark, Kafka, Sql, Apache Airflow, Spark Streaming, Kinesis, Terraform, Databricks, Python, Kubernetes, Prefect, DuckDB, dbt, Delta Lake
7-12 yrs
Chennai
Skills:
Pyspark, Data Engineer, Linux, Python, Jenkins
9-11 yrs
Chennai, India
Skills:
.NET, data engineering , C#, Metadata Management, Adf, Data Architecture, ELT, Nosql, Javascript, Data Lake, Python, Data Modelling, Sql, Devops, Data Quality, Data Governance, Databricks, Azure, Etl, lakehouse, data pipelines, model training, model deployment, agile delivery, warehouse design, Synapse, ML fundamentals, Azure Data Engineering, CI/CD
5-7 yrs
Chennai, India
Skills:
Kafka, API design, Data Modeling, Microservices, ELT, Distributed Systems, Python, Java, BigQuery, Hadoop, Data Quality, Hive, MLops, Spark, Data Governance, Databricks, Fault Tolerance, Kubernetes, Etl, Airflow, GenAI, LLM tooling, event-driven patterns, asynchronous processing, cloud-native practices, observability, Big Data ecosystem, HDFS, Delta Lake, CI/CD
7-9 yrs
Chennai, India
Skills:
Pyspark, Scala, Azure Databricks, Informatica, Sql, ELT, Apache Airflow, Azure Data Factory, Azure Data Lake, Python, Etl, Oracle PLSQL, Unity Catalog, CI CD pipelines, Delta Lake