Description
We are seeking an experienced Data Engineer to join our team in India. The ideal candidate will have a strong background in data engineering with expertise in Python, PySpark, MySQL, and Palantir. This role involves designing and maintaining robust data pipelines, collaborating with cross-functional teams, and ensuring high data quality standards.
Responsibilities
- Design, develop, and maintain data pipelines using Python, PySpark, and MySQL.
- Work with Palantir to integrate and analyze large datasets.
- Collaborate with data scientists and analysts to understand data requirements and ensure data availability.
- Optimize data models and improve data processing performance.
- Perform data quality checks and troubleshoot data issues.
- Document data processes and maintain data dictionaries.
Skills and Qualifications
- 12-20 years of experience in data engineering or related field.
- Proficiency in Python programming for data manipulation and analysis.
- Experience with PySpark for big data processing.
- Strong knowledge of MySQL for database management and querying.
- Familiarity with Palantir for data integration and visualization.
- Understanding of data warehousing concepts and ETL processes.
- Ability to work collaboratively in a team environment and communicate effectively.
