Azure Data Engineer

The Team

Deloitte's Technology & Transformation practice can help you uncover and unlock the value buried deep inside vast amounts of data. Our global network provides strategic guidance and implementation services to help companies manage data from disparate sources and convert it into accurate, actionable information that can support fact-driven decision-making and generate an insight-driven advantage. Our practice addresses the continuum of opportunities in business intelligence & visualization, data management, performance management and next-generation analytics and technologies, including big data, cloud, cognitive and machine learning.

We are seeking highly skilled Azure Databricks Data Engineers to join our data modernization team. You will play a pivotal role in designing, developing, and maintaining robust data solutions on the Databricks platform. Your experience in data engineering, along with a deep understanding of technology and tools, will be instrumental in building solutions to drive data-driven decision-making across a variety of customers.

Mandatory skills:

Databricks, PySpark, SQL, Azure Functions, Live / Streaming/batch processing data, Azure Data Factory (ADF), Logic Apps, ELT / ETL, Data Integration & Ingestion, Data Manipulation and Processing, GITHUB, Actions, Azure DevOps

Optional skills: Synapse,Data Modelling.

Experience in project management, running a scrum team.

Exposure to working with an external technical ecosystem.

Key Skills:

• Design and implement scalable, secure, and high-performance data architectures on Azure.
• Design, develop, and optimize ETL/ELT data pipelines using Azure Data Factory (ADF), Databricks, and Synapse Analytics.
• Implement data ingestion, transformation, and storage solutions using Azure Data Lake, Azure SQL, and Azure Synapse.
• Develop batch and real-time data processing solutions using Azure services like Delta Live Tables, Event Hubs, Stream Analytics, and Azure Functions.
• Development experience as a data engineer with a focus on core tools and technologies like ADF, Databricks, Key Vault, ADLS, SQL Server / Any RDMS, Fabric, Azure Synapse.
• Strong SQL and Python skills for data transformation and analysis.
• Design and implement data models, schemas, and performance-optimized queries.
• Work with structured, semi-structured, and unstructured data sources.
• Ensure data quality, governance, and security in all solutions.
• Deploy data solutions using Azure DevOps CI/CD pipelines.
• Work closely with data scientists, analysts, and business users to support data-driven decision-making.
• Troubleshoot and optimise data workflows for scalability and performance.
• Ensure data solutions are cost-optimised, scalable, and secure on Azure.
• Implement logging, monitoring, and alerting solutions using Azure Monitor and Log Analytics

Secondary skills:

• Project management – status reports, stand-ups, knowledge of Agile methodology
• Stakeholder management
• Mentor data engineers and analysts
• Provide technical direction and design reviews
• A determined focus on the user and user experience when problem- solving
• Team player and able to collaborate, proactively communicates and shares understanding and experience within a team
• Clear communicator and able to work in a multicultural/lingual environment
• Uses initiative and owns their deliverables end-to-end
• Brings ideas to the table, is inquisitive and excited by new technology
• Used to working in a flexible, independent manner on a mixture of small unstructured and large structured items
• Provides regular, considered feedback and regularly strives to improve their own ways of working

Soft Skills:

• Engage with business leaders, developers, infra, and security teams
• Understand business requirements deeply
• Manage expectations and negotiate priorities
• self-starter and proactive to provide cost and performance-oriented solutions.

Preferred Qualifications / Certifications

• Minimum 5+ years of experience
• Any graduation, B-tech, BE, MBA, BSC, MSC
• Experience working in regulated environments and with internal systems quality policies and procedures.
• Experience in development and deployment on cloud infrastructure.
• Azure Certifications (Azure Data Engineer Associate, Azure Solutions Architect).
• Databricks Certification