Role Title: Azure Data Engineer
BU/Segment: Healthcare Analytics
Responsibilities
- Design, develop, test, and deploy robust and scalable data pipelines using Azure Databricks, PySpark, and Spark SQL for processing large volumes of structured and unstructured data.
- Implement workflow orchestration using Azure Data Factory and Databricks Workflows to automate ETL/ELT processes.
- Implement and maintain CI/CD pipelines for Databricks notebooks, scripts, and configurations using Azure DevOps or GitHub Actions, promoting code across dev, test, and prod environments.
- Develop and maintain healthcare data models, including data entities, relationships, and data flows, to support healthcare analytics and decision-making.
- Design and implement enterprise-wide BI architectures. Develop and maintain data warehouses and data marts.
- Develop and maintain reports, dashboards, and data visualizations. Collaborate with stakeholders to understand business needs and provide data-driven insights
Person attributes
- 6+ years of experience as a Data Engineer with a proven track record of building data pipelines in the cloud.
- 3+ years of hands-on experience with Azure Databricks
- Strong experience with core Azure data services Azure Data Lake Storage (Gen 2), Azure data factory.
- Proficiency in Python and expert-level skills in SQL
- Solid understanding of data warehousing concepts, dimensional modeling (star/snowflake schemas), and data architecture principles
- Experience with CI/CD pipelines (Azure DevOps, Jenkins, GitHub Actions) and version control systems (Git).
- Demonstrated experience and knowledge of strategic problem solving and frameworks, and project management skills
- Excellent written and verbal communication with the ability to establish credibility and strong relationships with stakeholders
