We are seeking an experienced and strategic Data Architect to design, build, and optimize scalable, secure, and high-performance data solutions. You will play a pivotal role in shaping our data infrastructure, working with technologies such as Databricks, Azure Data Factory, Unity Catalog, and Spark, while aligning with best practices in data governance, pipeline automation, and performance optimization.
Responsibilities
- Design and develop scalable data pipelines using Databricks and Medallion Architecture (Bronze, Silver, and Gold layers).
- Architect and implement data governance frameworks using Unity Catalog and related tools.
- Write efficient PySpark and SQL code for data transformation, cleansing, and enrichment.
- Build and manage data workflows in Azure Data Factory (ADF), including triggers, linked services, and integration runtimes.
- Optimize queries and data structures for performance and cost-efficiency.
- Develop and maintain CI/CD pipelines using GitHub for automated deployment and version control.
This job was posted by Ana Bardhan from Nineleaps.
