AWS Data Engineer- Airflow

Years of Experience: 5-7 Years

Job Summary: We are seeking a highly motivated and detail-oriented Data Consultant who is flexible, proactive, and capable of working independently. The ideal candidate will possess strong communication and coordination skills, with a high level of diligence and attention to detail. This role requires collaboration with cross-functional teams to ensure seamless project execution while demonstrating strong ownership and the ability to work with minimal supervision.

Responsibilities

Collaborate effectively with cross-functional teams to ensure seamless project execution.
Proactively identify issues and implement timely solutions.
Maintain high standards of accuracy and attention to detail in all deliverables.
Utilize SQL and data warehousing concepts to manage and analyze data.
Work with Amazon Redshift and other AWS services including Glue Jobs, Step Functions, RDS, and SageMaker.
Develop and manage workflows using Airflow (DAGs).
Implement MLOps practices and utilize PySpark and Python for data processing.
Utilize version control tools such as GitHub, including Copilot, for project management.

Mandatory Skills

Strong experience in SQL and data warehousing concepts.
Hands-on experience with Amazon Redshift.
Expertise in workflow orchestration tools such as Airflow (DAGs).
Experience with AWS services including Glue Jobs, Step Functions, RDS, and SageMaker.
Proficiency in PySpark and Python.
Understanding of MLOps practices.
Familiarity with version control tools such as GitHub (including Copilot).

Preferred Skills

Experience with SSIS and Netezza.
Knowledge of BMO processes.
Familiarity with AWS CDK (Cloud Development Kit).

Qualifications

Bachelor's degree in computer science, Information Technology, or a related field.
5-7 years of relevant experience in data consulting or a similar role.