About The Role
We are seeking a skilled Data Engineer to join our team. The ideal candidate will have a solid background in data engineering, with expertise in AWS technologies and data processing frameworks.
Key Responsibilities
- Design and implement data pipelines using AWS services and PySpark.
- Develop and optimize SQL queries for data extraction and transformation.
- Monitor and maintain data workflows in Apache Airflow.
- Collaborate with data scientists and analysts to understand data requirements.
- Ensure data quality and integrity across all platforms.
- Document data architecture and engineering processes.
- Stay updated with emerging technologies and best practices in data engineering.
Required Qualifications
- 3-5 years of experience in data engineering or related field.
- Proficient in AWS services such as S3, Redshift, and Glue.
- Strong experience with PySpark for big data processing.
- Solid SQL skills for data manipulation and analysis.
- Experience with workflow orchestration using Apache Airflow.
Nice to have
- Familiarity with data warehousing concepts and architecture.
- Experience with ETL processes and tools.
- Knowledge of Python or Java programming for data processing.
- Understanding of data visualization tools and techniques.
