Job Title: AWS Data Engineer
Location: Pune, Bangalore, Delhi
Experience: 713 Years
Role Overview
We are seeking an experienced AWS Data Engineer to design, develop, and maintain robust data engineering solutions on AWS Cloud. The ideal candidate will have strong expertise in Python, PySpark, and AWS services, along with experience in data lake architecture and automation.
Required Technical Skill Set
- Core Skills: Python, Data Engineering, PySpark, AWS Cloud, CosmosDB
- Additional Skills: Terraform, Java (for SCADA), MQTT, Greengrass, OPC-UA, Windows Server, PowerShell, Grafana, SQL, AWS CloudWatch, AWS Glue
Responsibilities
- Design and implement data transformation workflows using Python and AWS services.
- Develop and maintain infrastructure using Terraform and AWS Cloud.
- Standardize and manage raw and curated data in a multi-source Data Lake.
- Build and maintain SCADA data collection software deployed on AWS Greengrass.
- Automate deployment of Greengrass across multiple sites and ensure operational stability.
- Monitor and manage Windows servers hosting Greengrass using PowerShell.
- Create and maintain Grafana dashboards for data visualization and system monitoring.
- Coordinate with IT teams to manage incidents and plan maintenance activities.
- Use AWS CloudWatch to monitor system health and report errors.
- Integrate CRM data (Salesforce) into Power BI using AWS Glue and Python Lambdas.
- Collaborate with Data Platform teams to manage access, API provisioning, and folder creation.
Desired Competencies
Technical Skills
- Proficiency in Python for data transformation and automation.
- Experience with Terraform and AWS Cloud infrastructure.
- Familiarity with Java for SCADA software development.
- Understanding of Data Lake architecture and data modeling.
- Knowledge of MQTT, Greengrass, and OPC-UA protocols.
- Experience with Windows Server management and PowerShell scripting.
- Ability to create and manage Grafana dashboards using SQL queries.
- Basic knowledge of AWS CloudWatch and AWS Glue.
Behavioral Skills
- Strong problem-solving and debugging skills.
- Proactive and detail-oriented approach to system monitoring and incident management.
- Effective communication with IT and data platform teams.
Soft Skills
- Strong analytical and problem-solving abilities.
- Clear communication in English.
- Ability to work independently and collaboratively in a cross-functional team.
- Strong focus on code quality, documentation, and collaboration.
Highly Recommended
- Advanced understanding of SCADA systems and industrial data flows.
- Experience with automated deployment and multi-site configuration management.
- Familiarity with network administration and cybersecurity principles.
- Basic knowledge of Salesforce integration.
