Job Overview
This role is crucial for driving the development and optimization of our data pipelines, ensuring data quality. The ideal candidate is a problem-solver who is proficient in Python, SQL, MongoDB and data orchestration tools.
Key Responsibilities
- Design, construct, install, test, and maintain highly scalable data pipelines.
- Develop complex database queries for data analysis and extraction.
- Implement processes and systems to monitor data quality, ensuring production data is always accurate and available for key stakeholders.
- Optimize data pipelines and architectures for performance and scalability.
- Integrate new data management technologies and software engineering tools into existing structures.
- Work with data and analytics experts to strive for greater functionality in our data systems.
- Experience with Git and platforms like GitHub/GitLab for source code management.
Qualifications
- Bachelor's or Master's degree in Computer Science, Information Technology, or are related field.
- 3+ years of experience in a Data Engineer role or similar position.
- Advanced proficiency in Python programming for data engineering tasks.
- Expertise in SQL database design and development.
- Strong experience with data modeling and building ETL pipelines.
- Proven experience with orchestration tools such as Apache Airflow, Prefect, or similar.
- Familiarity with cloud services like AWS or GCP.
- Excellent analytical and problem-solving abilities.
- Strong communication and collaboration skills, able to work effectively with teams across the organization
(ref:hirist.tech)
