As an Associate Data Engineer Graph, you will design, build, and maintain data lake solutions for scientific research data. This role involves building scalable and high-performance data engineering solutions for large datasets and collaborating with research scientists and other stakeholders. The ideal candidate has strong technical skills, experience with big data technologies, and an understanding of data architecture and ETL processes.
Roles & Responsibilities
- Data Engineering & ETL: Design, develop, and implement data pipelines, ETL/ELT processes, and data integration solutions. You'll contribute to data pipeline projects from inception to deployment, managing scope, timelines, and risks.
- Data Modeling & Optimization: Contribute to data models and data dictionaries to ensure data accuracy and consistency. You will also be responsible for optimizing large datasets for query performance.
- Collaboration & Innovation: Collaborate with global, cross-functional teams, including research scientists, to understand data requirements and design solutions. You will also explore new tools and technologies to improve the ETL platform's performance.
- Cloud & Security: Leverage cloud platforms (AWS preferred) to build scalable and efficient data solutions. You will also implement data security and privacy measures to protect sensitive data.
- Documentation & Best Practices: Adhere to best practices for coding, testing, and designing reusable code. You will also maintain documentation of processes, systems, and solutions in tools like Confluence.
Qualifications
- A Bachelor's degree with 0-3 years of experience or a Diploma with 4-7 years of experience in Computer Science, IT, or a related field.
- Proficiency in SQL and Python for data engineering and scripting tasks.
- Hands-on experience with big data technologies like Databricks and Apache Spark (PySpark, SparkSQL).
- A strong understanding of data modeling, data warehousing, and data integration concepts.
- Experience with relational databases (Oracle, MySQL, SQL Server, PostgreSQL).
- Knowledge of cloud data platforms (AWS preferred).
- Experience with data visualization tools (Dash, Plotly, Spotfire) is a plus.
- Databricks Certified Data Engineer Professional certification is preferred.
Soft Skills
- Problem-Solving: Excellent critical-thinking and problem-solving skills, with the ability to work with large, complex datasets.
- Communication: Strong communication and collaboration skills, with demonstrated awareness of how to function in a team setting.
- Initiative: A high degree of initiative and self-motivation, with a passion for tackling complex challenges in drug discovery.
- Adaptability: The ability to learn new tools and technologies to improve platform performance.
