Description
We are seeking an experienced PySpark Developer to join our dynamic team in India. The ideal candidate will have a strong background in data engineering and a passion for building scalable data solutions.
Responsibilities:
- Develop and maintain data processing pipelines using PySpark, SQL, and Hadoop.
- Collaborate with data scientists and analysts to optimize data workflows.
- Implement data transformation and aggregation processes.
- Ensure secure data access and compliance with data governance policies.
- Perform Spark job tuning and performance optimization.
- Write unit tests and documentation for Spark transformations.
- Work on continuous improvement of data processing frameworks.
Skills Required:
- Proficiency in PySpark, SQL, and Hadoop.
- Experience with big data technologies and frameworks.
- Strong problem-solving and analytical skills.
- Ability to work in a collaborative team environment.
- Excellent communication skills.
