Cloud Data Engineer

Roles and Responsibilities:

1. Data Pipeline and Analytics Development:

• Design, implement, and manage scalable data pipelines and analytics solutions using AWS services (Glue, Redshift) and Azure tools (Data Factory, Synapse).

1. Cloud IoT Services Expertise:

• Work with IoT cloud services, including AWS IoT Core and Azure IoT Hub, to collect, process, and analyze IoT data efficiently.

1. Data Integration:

• Develop solutions for integrating data from diverse sources, ensuring real-time or batch processing as per business requirements.

1. Performance Optimization:

• Optimize performance of cloud-based data solutions to handle large-scale datasets effectively.

1. Troubleshooting and Debugging:

• Identify and resolve issues in data pipelines, ensuring high availability and reliability of cloud data platforms.

1. Collaboration:

• Work closely with data analysts, data scientists, and business stakeholders to meet data needs and deliver insights.

1. Documentation and Standards Compliance:

• Maintain comprehensive documentation of data architecture, processes, and workflows while ensuring compliance with best practices and security standards.

Requirements:

• Strong expertise in AWS Glue, AWS Redshift, Azure Data Factory, and Azure Synapse Analytics.
• Hands-on experience with IoT cloud platforms, such as AWS IoT Core and Azure IoT Hub.
• Proficiency in building and managing data pipelines.
• Knowledge of cloud security practices and cost optimization strategies.
• Excellent problem-solving and communication skills.