AWS Data Engineer

Key Responsibilities:

• Strong hands on AWS Data Engineering experience across large scale cloud data platforms
• Expertise in AWS S3 AWS Glue AWS Lambda AWS Redshift AWS Lake Formation AWS CloudWatch and AWS IAM
• Strong experience in real time and streaming data processing using Amazon Kinesis Data Streams Kinesis Firehose Kinesis Analytics Amazon Kafka or equivalent streaming technologies
• Hands on coding proficiency in Python PySpark and SQL with strong optimization skills
• Experience building high volume low latency data ingestion and transformation pipelines
• Strong experience in Databricks Delta Lake Structured Streaming and Spark Performance Tuning
• Experience handling TB PB scale datasets partitioning strategies file optimization schema evolution and data lifecycle management
• Strong understanding of Data Lake Lakehouse and Modern Data Platform architectures
• Practical data modeling experience for analytical operational and reporting workloads
• Experience implementing CI CD pipelines using AWS CodePipeline CodeBuild CodeDeploy GitHub Actions or Jenkins
• Strong troubleshooting skills for production issues performance bottlenecks and cost optimization
• Excellent client communication stakeholder management and problem solving skills

Technical Requirements:

• Responsibilities and JD in brief along with additional criteria to be considered
• Design and develop scalable AWS Data Engineering solutions and cloud native data platforms
• Build batch and real time data pipelines using AWS Glue PySpark Databricks S3 Kinesis Redshift and Kafka MSK
• Develop and optimize Databricks Delta Lake and Spark workloads for large scale data processing with Pyspark Databricks transformations
• Implement streaming solutions using Amazon Kinesis Data Streams Firehose and Analytics for near real time data ingestion and processing
• Handle high volume TB PB scale data pipelines with a focus on performance scalability and cost optimization
• Develop complex SQL transformations data models and analytics ready datasets
• Implement CI CD Infrastructure as Code Terraform CloudFormation and DevOps best practices
• Ensure data quality governance security monitoring and production support
• Collaborate with architects stakeholders and cross functional teams in Agile delivery environments
• Troubleshoot and optimize end to end AWS data platforms streaming pipelines and cloud infrastructure

Preferred Skills:

Technology->Cloud->AWS Core services,Technology->Cloud->AWS - Database Technologies->AWS,Technology->Cloud Platform->AWS Data Analytics