Job Description
Strong hands-on experience with Azure Databricks Proficient in Apache Spark (PySpark/Scala) Experience with Azure Data Lake (ADLS Gen2) and cloud storage Strong SQL skills and experience with large datasets Experience with Azure Data Factory (ADF) for orchestration Knowledge of data modeling, partitioning, and performance tuning Experience working with big data frameworks
Design, develop, and maintain data pipelines using Azure Databricks Implement ETL/ELT workflows to process large-scale structured and unstructured data Optimize Spark jobs for performance, scalability, and cost Work with Azure Data Lake Storage (ADLS Gen2), Azure Blob Storage, and Azure SQL Develop and manage Databricks notebooks using PySpark / Scala / SQL Integrate Databricks with Azure services such as: Azure Data Factory Azure Synapse Analytics Azure Key Vault Implement data quality, validation, and monitoring frameworks
