Senior Data Engineer (PySpark, SQL)

*Job Title:** Senior Data Engineer – PySpark, SQL

**Location:** Gurugram / Chennai

**Employment Type:** Full-time

**Experience:** 10+ Years

**Notice Period:** Immediate Joiner or maximum 7 days

**Mandatory Certification:** Active Azure Databricks Certification

### Job Overview

We are looking for an experienced Senior Data Engineer with strong hands-on expertise in PySpark, SQL, Python, and Azure Databricks. The candidate will be responsible for designing, developing, and optimizing scalable data pipelines and ETL/ELT processes to support analytics, reporting, and machine learning applications.

### Key Responsibilities

- Design, develop, and maintain scalable ETL/ELT data pipelines.

- Build and optimize large-scale data processing solutions using PySpark and Apache Spark.

- Write complex SQL queries, perform query tuning, and optimize database performance.

- Develop and maintain data lakes, data warehouses, and efficient data models.

- Work with Azure Databricks, Azure Data Lake, and Azure Synapse.

- Automate data workflows using Azure Data Factory, Apache Airflow, or similar tools.

- Implement data quality checks, validation, monitoring, and governance practices.

- Troubleshoot production pipeline issues and ensure reliable data delivery.

- Collaborate with data scientists, analysts, and business stakeholders.

- Conduct code reviews, follow CI/CD and Git practices, and mentor junior engineers.

### Mandatory Skills

- 10+ years of overall experience in data engineering.

- Strong hands-on experience with **PySpark and Advanced SQL**.

- **Active Azure Databricks certification is mandatory.**

- Strong Python programming and scripting skills.

- Experience designing and implementing enterprise-level ETL/ELT pipelines.

- Knowledge of distributed systems, big data architecture, data lakes, and data warehousing.

- Experience with SQL performance tuning, complex joins, and window functions.

- Immediate joiner or maximum 7 days notice period.

### Good-to-Have Skills

- Azure Data Factory, Azure Synapse, Azure Key Vault.

- Apache Airflow or other workflow orchestration tools.

- Kafka, Spark Structured Streaming, or Azure Event Hubs.

- Data governance, data cataloging, and data security.

- Exposure to Snowflake, Amazon Redshift, AWS, or Google BigQuery.

### Education

Bachelor's or Master's degree in Computer Science, Engineering, or a related field.ring, or a related field.