Aws Developer

Job Description:

• We are looking for an experienced AWS Glue Developer Data Engineer to design develop and optimize scalable ETL ELT data pipelines on AWS
• The candidate should have strong hands on experience with AWS Glue Python PySpark SQL data lake technologies and AWS data services

Key Responsibilities:

• Design develop and maintain ETL ELT pipelines using AWS Glue
• Develop AWS Glue jobs using Python PySpark and SQL for large scale data processing
• Build data ingestion and transformation pipelines across structured and semi structured data sources
• Configure and manage AWS Glue Data Catalog Jobs Workflows and Triggers
• Work with AWS services such as Amazon S3 Redshift and IAM
• Implement data transformations validations data quality checks and reconciliation processes
• Optimize Glue Spark jobs for performance scalability reliability and cost efficiency
• Implement incremental loads partitioning strategies error handling restart recovery and audit frameworks
• Troubleshoot ETL pipeline failures and production performance issues
• Implement appropriate security controls including IAM roles encryption and access management
• Support migration of legacy mainframe or on premises ETL workloads to the AWS cloud data platform
• Develop automated deployment processes using CI CD and Infrastructure as Code practices where applicable
• Collaborate with data architects analysts business teams and cloud platform teams to translate requirements into scalable data solutions
• Prepare technical design documents mapping specifications operational documentation and deployment runbook documentation

Technical Requirements:

• Primary skills Technology Cloud Platform AWS Enviornment Provisioning Technology Data On Cloud NoSQL Amazon Dynamo DB Technology Data On Cloud Platform AWS

Additional Responsibilities:

• Technical Skills
• AWS Glue
• Python PySpark Apache Spark
• SQL
• Amazon S3
• Data Lake Cloud Data Warehouse concepts
• ETL ELT design and development
• Performance optimization and troubleshooting
• Experience
• 5 years of overall Data Engineering ETL development experience
• Strong hands on experience developing production grade AWS Glue and PySpark solutions
• Experience handling high volume datasets and complex transformations in AWS environments
• Preferred Certification
• AWS Certified Data Engineer Associate or other relevant AWS certification

Preferred Skills:

Technology->Cloud Platform->AWS Core services,Technology->Cloud Platform->AWS Database,Technology->Data On Cloud - Platform->AWS