Job Description
Experience (6 - 9 years)
We are seeking a highly skilled Data Engineer with a strong background
, Databricks, Python, Microsoft Fabric, Azure Data Stack, and SQL. The ideal candidate will have extensive experience in automating data pipelines and managing complex data integration scenarios using Fabric/Databricks and is known for taking initiative and delivering top-quality work and documentation.
Key Responsibilities
- Develop and maintain scalable data pipelines and data integrations to support continuing increases in data volume and complexity.
- Utilize Databricks, Python, SQL for data extraction, transformation, and loading (ETL) in complex scenarios.
- Collaborate with stakeholders and agile teams to understand and address complex data requirements.
- Deliver top-quality work and documentation, demonstrating a strong attention to detail and a commitment to accuracy and clarity.
- Implement robust measures to ensure data accuracy, quality, and accessibility.
- Monitor data performance, identify bottlenecks, and modify infrastructure as needed.
- Work at a high pace in an agile team environment, taking initiative to solve problems and improve systems.
Technical Skills
- Microsoft Fabric
- Azure Data Factory
- Azure Databricks
- Azure Functions
- Azure DevOps including CI/CD pipelines
- Good understanding of Infrastructure as Code
- Data modelling
- SQL
