AWS Data Engineer

Key Responsibilities

• Design, develop, and maintain scalable data pipelines using AWS cloud services.
• Build and optimize data processing workflows using Python, PySpark, and Apache Spark.
• Develop and manage enterprise data warehouses and data marts using Amazon Redshift.
• Implement real-time and streaming data ingestion pipelines using Apache Kafka and MQTT.
• Develop and manage ETL/ELT workflows for integrating data from multiple sources.
• Implement workflow orchestration and scheduling using Apache Airflow.
• Design and maintain analytical data models to support business intelligence and reporting.
• Write complex SQL queries and optimize database performance.
• Work with cloud-native AWS technologies to build reliable, scalable, and high-performance data solutions.
• Ensure data quality, consistency, security, and pipeline reliability.
• Collaborate with architects, analysts, and cross-functional teams to deliver data engineering solutions.

Required Skills & Technical Expertise

• Strong hands-on experience in AWS Data Engineering.
• Good experience with Python and PySpark programming.
• Strong knowledge of Apache Spark and distributed data processing.
• Experience with Amazon Redshift, enterprise data warehouses, and data marts.
• Hands-on experience with Apache Kafka and MQTT for data streaming and ingestion.
• Experience in Apache Airflow for workflow orchestration and pipeline scheduling.
• Strong SQL skills, including query writing and performance optimization.
• Experience in ETL/ELT development and analytical data modeling.
• Understanding of cloud-native architecture and scalable data pipeline design.

Good to Have

• Experience with AWS services such as S3, Glue, Lambda, EMR, Kinesis, and CloudWatch.
• Knowledge of data lake and data lakehouse architectures.
• Experience with real-time data processing and event-driven architectures.
• Understanding of CI/CD, Git, and deployment automation.
• Experience with data governance, data quality, and monitoring frameworks.