Job Description

AWS Glue Data Engineer

We are seeking an experienced AWS Glue Data Engineer to join our dynamic team. This role is critical in managing and automating data pipelines using AWS Glue, ensuring data integrity and availability for our analytics and reporting needs. The ideal candidate will have a strong background in data engineering and a deep understanding of AWS services to build scalable and efficient data solutions. The right person will be responsible for designing, implementing, and maintaining data workflows that drive our business decisions.

Key Responsibilities:

• Design, develop, and maintain robust AWS Glue ETL jobs to transform and load data.
• Automate data extraction, transformation, and loading processes to improve data pipeline efficiency.
• Collaborate with data scientists and analysts to understand data requirements and optimize data workflows.
• Ensure data quality and integrity through validation checks and error handling in data pipelines.
• Optimize AWS Glue job performance and cost-effectiveness by tuning job configurations.
• Develop and maintain documentation for data pipelines and processes to ensure team alignment.
• Troubleshoot and resolve data pipeline issues to maintain continuous data flow.
• Participate in code reviews and provide technical guidance to junior team members.
• Stay updated with the latest AWS services and industry best practices to enhance data solutions.
• Integrate AWS Glue with other AWS services to build comprehensive data architectures.

Qualification Criteria:

• Proven experience as a Data Engineer or similar role with a focus on AWS Glue.
• Strong understanding of AWS services, particularly AWS Glue, S3, and RDS.
• Experience with SQL and data modeling techniques.
• Proficiency in Python and scripting for ETL processes.
• Knowledge of data warehousing and data lake concepts.
• Excellent problem-solving skills and attention to detail.
• Ability to work in a fast-paced, collaborative environment.
• Strong communication skills to liaise with cross-functional teams.
• Experience with version control systems like Git.
• Familiarity with data governance and compliance requirements.

Primary Skills:

• AWS Glue
• ETL
• Python

Key Skills:

• AWS services
• SQL
• Data modeling
• Data warehousing
• Data lakes
• Version control
• Cloud computing
• Data governance
• API development
• Data security
• Data architecture
• Data visualization
• Machine learning
• Big data technologies
• Data integration
• DevOps
• Cloud architecture
• Data analytics
• Performance tuning
• Cloud security