Databricks

Responsibilities:

• Design and implement scalable data platforms and pipelines using Azure Databricks, Apache Spark, Delta Lake, and MLflow.
• Lead the migration from legacy platforms to Lakehouse architecture.
• Develop batch and streaming data pipelines for ingestion, transformation, and analytics.
• Establish standards for data governance, quality, and security.
• Collaborate with stakeholders to align architecture with business goals.
• Mentor data engineers and developers on Databricks best practices.
• Integrate Databricks with tools like Power BI, Tableau, Kafka, Snowflake, and Azure Data Factory.

Mandatory Skills:

• Strong command of Databricks and Azure.
• Proficiency in SQL, Python, Scala, and Spark.
• Experience with CI/CD pipelines, DevOps, and orchestration tools like Airflow or Data Factory.
• Familiarity with Azure cloud platforms.
• Deep understanding of distributed computing, performance tuning, and data security.

Preferred Skills:

• Experience with data mesh, data fabric, or enterprise data architectures.
• Strong ERP knowledge (e.g., SAP, Salesforce).
• Hands-on capability and self-exploration skills to drive analytics adoption across business teams.