Roles and Responsibilities:
- Data Pipeline and Analytics Development:
- Design, implement, and manage scalable data pipelines and analytics solutions using AWS services (Glue, Redshift) and Azure tools (Data Factory, Synapse).
- Cloud IoT Services Expertise:
- Work with IoT cloud services, including AWS IoT Core and Azure IoT Hub, to collect, process, and analyze IoT data efficiently.
- Data Integration:
- Develop solutions for integrating data from diverse sources, ensuring real-time or batch processing as per business requirements.
- Performance Optimization:
- Optimize performance of cloud-based data solutions to handle large-scale datasets effectively.
- Troubleshooting and Debugging:
- Identify and resolve issues in data pipelines, ensuring high availability and reliability of cloud data platforms.
- Collaboration:
- Work closely with data analysts, data scientists, and business stakeholders to meet data needs and deliver insights.
- Documentation and Standards Compliance:
- Maintain comprehensive documentation of data architecture, processes, and workflows while ensuring compliance with best practices and security standards.
Requirements:
- Strong expertise in AWS Glue, AWS Redshift, Azure Data Factory, and Azure Synapse Analytics.
- Hands-on experience with IoT cloud platforms, such as AWS IoT Core and Azure IoT Hub.
- Proficiency in building and managing data pipelines.
- Knowledge of cloud security practices and cost optimization strategies.
- Excellent problem-solving and communication skills.
