Job Summary
We are seeking an experienced Azure Data Engineer with strong hands-on expertise in building, optimizing, and monitoring data pipelines on Microsoft Azure. The ideal candidate should have solid experience with Grafana for monitoring, visualization, and performance tracking of data platforms and cloud infrastructure.
Key Responsibilities
- Design, develop, and maintain scalable data pipelines using Azure services.
- Build and optimize ETL/ELT workflows using Azure Data Factory, Databricks, and Azure Synapse Analytics.
- Work with structured and semi-structured data from multiple sources.
- Implement data processing using PySpark / Spark SQL.
- Develop and manage data models for analytics and reporting.
- Monitor data pipelines and platform health using Grafana dashboards.
- Integrate Grafana with Azure Monitor, Log Analytics, and other data sources.
- Troubleshoot performance issues and optimize data workflows.
- Collaborate with DevOps, BI, and application teams.
- Ensure data quality, security, and compliance standards.
Required Skills & Qualifications
- 5+ years of experience as an Azure Data Engineer.
- Strong hands-on experience with:
- Azure Data Factory (ADF)
- Azure Databricks
- Azure Synapse Analytics
- Azure Data Lake Storage (ADLS Gen2)
- Proficiency in Python and PySpark.
- Strong SQL skills.
- Experience with Grafana:
- Dashboard creation
- Metrics visualization
- Alerts and monitoring
- Experience integrating Grafana with Azure Monitor / Log Analytics.
- Knowledge of cloud data architecture and best practices.
Good to Have
- Experience with CI/CD pipelines (Azure DevOps).
- Knowledge of Terraform / ARM templates.
- Exposure to Snowflake or other cloud data warehouses.
- Basic knowledge of DevOps & monitoring tools (Prometheus, Azure Monitor).
- Azure certifications (DP-203 preferred).
Soft Skills
- Strong problem-solving and analytical skills.
- Good communication and stakeholder coordination.
- Ability to work independently and in a team environment.
