Pyspark Developer

Description

We are seeking an experienced PySpark Developer to join our dynamic team in India. The ideal candidate will have a strong background in data engineering and a passion for building scalable data solutions.

Responsibilities:

• Develop and maintain data processing pipelines using PySpark, SQL, and Hadoop.
• Collaborate with data scientists and analysts to optimize data workflows.
• Implement data transformation and aggregation processes.
• Ensure secure data access and compliance with data governance policies.
• Perform Spark job tuning and performance optimization.
• Write unit tests and documentation for Spark transformations.
• Work on continuous improvement of data processing frameworks.

Skills Required:

• Proficiency in PySpark, SQL, and Hadoop.
• Experience with big data technologies and frameworks.
• Strong problem-solving and analytical skills.
• Ability to work in a collaborative team environment.
• Excellent communication skills.