Core Skills
- PySpark – mid-level knowledge (DataFrames, Spark SQL, basic transformations)
- Notebook creation – organizing and collaborating in Databricks
- Data ingestion – loading data from multiple sources
- Data modeling – designing tables and schemas
- Medallion architecture – Bronze → Silver → Gold layers
Databricks Platform
- Workspace & notebooks – collaborative development
- Jobs & workflows – scheduling and automation
- Delta Lake – ACID-compliant storage
- Unity Catalog – data governance and access control
DevOps / Automation
- CI/CD pipelines – Git, GitHub Actions, Azure DevOps
ADF - Azure Data Factory – pipeline creation, scheduling, and monitoring
