ETL Development & Data Pipeline Management
- Develop, maintain, and optimize ETL pipelines using AWS native services.
- Design scalable and efficient data pipelines for data ingestion, transformation, and processing.
- Write complex SQL queries and Python scripts to support data transformation and automation.
- Ensure high reliability and performance of data workflows across systems.
AWS Data Engineering
- Work extensively with AWS services including AWS Glue, Lambda, and AWS Data Catalog.
- Develop and manage ETL jobs, crawlers, and metadata management using AWS Glue.
- Implement serverless data processing solutions using AWS Lambda.
- Ensure scalable and secure cloud data architecture within the AWS environment.
Production Support & Monitoring
- Monitor live data pipelines and troubleshoot issues affecting performance or data accuracy.
- Provide production support by identifying root causes and implementing fixes.
- Ensure data pipeline stability through proactive monitoring and maintenance.
Data Collaboration & Integration
- Collaborate with analysts, cloud engineers, and business teams to ensure accurate data delivery.
- Support integration of data systems and maintain consistency across platforms.
- Assist in building data workflows that support analytics and reporting needs.
Data Quality, Security & Optimization
- Implement best practices for data quality validation and monitoring.
- Ensure data security and compliance within cloud data environments.
- Optimize AWS resources for performance and cost efficiency.
Documentation & Process Improvement
- Maintain documentation for data pipelines, workflows, and system architecture.
- Contribute to automation and process improvement initiatives.
- Support agile development processes and participate in continuous improvement activities.
