Aws Developer

Responsibilities :

Design, develop, and maintain ETL/ELT pipelines using AWS Glue.

Develop AWS Glue jobs using PySpark, and SQL for large-scale data processing.

Build data ingestion and transformation pipelines across structured and semi-structured data sources.

Configure and manage AWS Glue Data Catalog, Jobs, Workflows, and Triggers.

Work with AWS services such as Amazon S3, Redshift, and IAM.

Implement data transformations, validations, data-quality checks, and reconciliation processes.

Optimize Glue/Spark jobs for performance, scalability, reliability, and cost efficiency.

Implement incremental loads, partitioning strategies, error handling, restart/recovery, and audit frameworks.

Troubleshoot ETL pipeline failures and production performance issues.

Implement appropriate security controls including IAM roles, encryption, and access management.

Support migration of legacy/mainframe or on-premises ETL workloads to the AWS cloud data platform.

Develop automated deployment processes using CI/CD and Infrastructure-as-Code practices where applicable.

Collaborate with data architects, analysts, business teams, and cloud/platform teams to translate requirements into scalable data solutions.

Prepare technical design documents, mapping specifications, operational documentation, and deployment/runbook documentation.

Additional Responsibilities:

Technical Skills

AWS Glue

Python/PySpark / Apache Spark

SQL

Amazon S3

Data Lake / Cloud Data Warehouse concepts

ETL/ELT design and development

Performance optimization and troubleshooting Experience

7+ years of overall Data Engineering / ETL development experience.

Strong hands-on experience developing production-grade AWS Glue,Python and PySpark solutions.

Experience handling high-volume datasets and complex transformations in AWS environments. Preferred Certification

AWS Certified Data Engineer †Associate or other relevant AWS certification.