AWS Data engineer

Job Overview

We are seeking an experienced Cloud Data Engineer to the architecture, development, Data migration,

ETL pipeline in cloud and optimization of a next-generation data platform. This is a development role for

an individual with over 4 years of deep data engineering and data migration, data pipeline in AWS

expertise, capable of driving technical direction, and delivering high-impact solutions in a fast-paced

Key Responsibilities

• Platform Development
• Knowledge of best practices for our end-to-end data pipelines in Cloud, ensuring
  
  

scalability, reliability, and security across the platform.

• Aggressively folowing the project timeline to ensure rapid delivery of data migratation
  
  

in AWS RDS.

• Working with Data Analysts, and business stakeholders to translate complex
  
  

requirements into robust, production-ready ETL data solutions.

• Data Pipeline Development
• Ingestion & Transformation: Design, build, and optimize high-volume data ingestion and
  
  

transformation jobs using tools like dbt Core, iceberg, AWS Glue, ensuring data quality

and integrity.

• Building ELT/ETL pipelines and using workflow orchestration tools like Apache Airflow or
  
  

dbt

• Optimize large-scale data processing algorithms and cloud data warehouse performance.
• Workflow: Develop and maintain sophisticated data pipelines using AWS glue/lambda,
  
  

focusing on modularity and reusability.

• Streaming & Real-time Integration: Implement and manage real-time data flows utilizing
  
  

native AWS streaming services (e.g., Kinesis) for immediate data availability.

• Data Security and Privacy: Data Anonymization, Compliance with Regulations
• Good knowledge of CI/CD for ETL pipeline automation and Data
  
  

migration

• Drive the Data Ecosystem Management & Monitoring
• Open Table Formats & Management: Implement and maintain the Iceberg open table
  
  

format, utilizing tools for efficient schema evolution and data management.

• Compute Engine Optimization: Optimize query performance and cost efficiency across
  
  

our primary compute engines: Amazon Redshift, and AWS Athena.

• rapidly identify bottlenecks, and troubleshoot production issues
  
  

Candidate Profile

• 4+ Years of hands-on, progressive experience in Cloud Data Engineering, cloud ecosystems AWS,
  
  

Data Architecture

• Deep conceptual understanding of core data engineering principles, ETL/ELT patterns, and
  
  

metadata management

• Ability to translate complex business needs into actionable technical requirements.
• Strong troubleshooting and problem-solving skills in distributed systems.
• Agile/Scrum teamwork capabilities and version control practices
• Insurance industry experience preferred but not mandatory
• Tools:
• Cloud Environment: AWS (S3, IAM, VPC, Glue, Lambda etc.)
• Experience with dbt Core, Iceberg, AWS Glue Catalog, AWS lambda, Redshift, Athena,
  
  

Splunk, AmazonQ, AWS streaming services, Git

• Strong SQL, Pyspark and Python
  
  

Skills: pipeline,etl,aws,dbt,data,management,cloud