AWS Data Engineer- Airflow
Years of Experience: 5-7 Years
Job Summary: We are seeking a highly motivated and detail-oriented Data Consultant who is flexible, proactive, and capable of working independently. The ideal candidate will possess strong communication and coordination skills, with a high level of diligence and attention to detail. This role requires collaboration with cross-functional teams to ensure seamless project execution while demonstrating strong ownership and the ability to work with minimal supervision.
Responsibilities
- Collaborate effectively with cross-functional teams to ensure seamless project execution.
- Proactively identify issues and implement timely solutions.
- Maintain high standards of accuracy and attention to detail in all deliverables.
- Utilize SQL and data warehousing concepts to manage and analyze data.
- Work with Amazon Redshift and other AWS services including Glue Jobs, Step Functions, RDS, and SageMaker.
- Develop and manage workflows using Airflow (DAGs).
- Implement MLOps practices and utilize PySpark and Python for data processing.
- Utilize version control tools such as GitHub, including Copilot, for project management.
Mandatory Skills
- Strong experience in SQL and data warehousing concepts.
- Hands-on experience with Amazon Redshift.
- Expertise in workflow orchestration tools such as Airflow (DAGs).
- Experience with AWS services including Glue Jobs, Step Functions, RDS, and SageMaker.
- Proficiency in PySpark and Python.
- Understanding of MLOps practices.
- Familiarity with version control tools such as GitHub (including Copilot).
Preferred Skills
- Experience with SSIS and Netezza.
- Knowledge of BMO processes.
- Familiarity with AWS CDK (Cloud Development Kit).
Qualifications
- Bachelor's degree in computer science, Information Technology, or a related field.
- 5-7 years of relevant experience in data consulting or a similar role.
