Data Engineer

Role: Data Engineer ( PySpark / SQL)

Client: AMAT (Applied Materials)

Experience: 5+ Years

Role: Data Engineer

Location: Bangalore

Mandatory Skills

• Python
• PySpark
• Apache Spark
• SQL

Job Summary

We are looking for an experienced Data Engineer with strong hands-on experience in Python, PySpark, Apache Spark and SQL to develop and maintain scalable data processing solutions.

Key Responsibilities

• Develop and maintain data processing pipelines using Python and PySpark.
• Work with large datasets using Apache Spark and Spark SQL.
• Write and optimize complex SQL queries.
• Perform data extraction, transformation and loading (ETL/ELT).
• Work on Spark transformations, actions, joins and aggregations.
• Optimize Spark jobs for better performance and scalability.
• Ensure data quality and reliability of data pipelines.
• Troubleshoot and resolve data processing issues.

Required Skills

• Strong hands-on experience in Python and PySpark.
• Good understanding of Apache Spark architecture and concepts.
• Strong SQL skills, including joins, CTEs and window functions.
• Experience in DataFrames, transformations and actions.
• Hands-on experience in developing ETL/ELT pipelines.
• Basic working knowledge of Databricks.

Preferred Skills

• Databricks notebooks, clusters and workflows.
• Delta Lake.
• Spark performance tuning and optimization.
• Azure or AWS cloud platforms.
• Experience with production data pipelines.

Ideal Candidate

A Data Engineer with approximately 5+ years of experience, strong practical knowledge of PySpark, Apache Spark and SQL, and basic exposure to Databricks.