Azure Databricks SME
Subject Matter Expert Role for Recruitment
Key Responsibilities
- Design and implement scalable data pipelines using Azure Data Bricks.
- Develop ETL processes to efficiently extract, transform, and load data.
- Collaborate with data scientists and analysts to define and refine data requirements.
- Optimize Spark jobs for performance and efficiency.
- Monitor and troubleshoot production workflows and jobs.
- Implement data quality checks and validation processes.
- Create and maintain technical documentation related to data architecture.
- Conduct code reviews to ensure best practices are followed.
- Work on integrating data from various sources including databases, APIs, and third-party services.
- Utilize SQL and Python for data manipulation and analysis.
- Collaborate with DevOps teams to deploy and maintain data solutions.
- Stay updated with the latest trends and updates in Azure Data Bricks and related technologies.
- Facilitate data visualization initiatives for better data-driven insights.
- Provide training and support to team members on data tools and practices.
- Participate in cross-functional projects to enhance data sharing and access.
- Strong expertise in Azure Data Bricks and data lake concepts.
- Proficiency with SQL, Python, and Spark.
- Solid understanding of data warehousing concepts.
- Experience with ETL tools and frameworks.
- Familiarity with cloud platforms such as Azure, AWS, or Google Cloud.
- Familiarity with Azure security features (Key Vault, Managed Identity, RBAC)
