Data Engineering Architect (ATC)

Job description

• Experience 10 years of experience in data engineering, ETL processes, and cloud based data solutions.
• AWS Expertise
• Hands-on experience with Amazon S3 for data storage, management, and retrieval.
• Proficient in AWS Athena for serverless querying and data analysis.
• Experience with AWS Glue for ETL jobs and data cataloging.
• Familiarity with AWS Data Pipeline or similar orchestration tools.
• Programming: Proficiency in SQL, Python, or other data engineering programming languages.
• Data Engineering Tools: Familiarity with tools like Apache Spark, Apache Airflow, or similar data orchestration and processing frameworks.
• Database Management: Experience with relational databases (MySQL, PostgreSQL) and NoSQL databases DynamoDB, Redshift.
• Big Data Experience with large-scale data processing and optimization for big data workloads.
• Cloud Technologies: Knowledge of cloud infrastructure, networking, and security best practices.

Analytical Skills

• Strong problem-solving skills and the ability to work with complex datasets.
• Communication: Excellent communication skills and ability to work collaboratively across teams.
• Design, build, and maintain scalable and efficient data pipelines using AWS technologies (S3, Athena, Glue, Lambda, etc.
• Architect and optimize data storage solutions on Amazon S3, ensuring cost effectiveness and efficient retrieval of large datasets.
• Develop automated data workflows using AWS Data Pipeline or similar orchestration tools for seamless data movement and processing.
• Monitor and troubleshoot data pipeline performance, ensuring high availability, reliability, and security
• Stay up to date with emerging technologies and trends in cloud data engineering and AWS services.