Senior Data Engineer

Company Description

At iPath Infotech, we assist businesses around the globe with top-notch IT solutions. We specialize in mobile app development, web development, BPO, software development, and managed services. Our team is dedicated to delivering exceptional results tailored to meet client needs. With a focus on affordable, high-quality solutions, we ensure that every project is a success. Explore our comprehensive services covering mobile apps, websites, business processes, and more to help you thrive in the digital era.

Role Description

This is a full-time on-site role for a Senior Data Engineer based in Ahmedabad. We are seeking a skilled and motivated data engineer to join our growing data team. The ideal candidate will have strong experience in Microsoft Fabric and be responsible for building, optimising and maintaining scalable data pipelines and architectures. You will work closely with data analysts and software engineers to ensure reliable data availability and efficient data processing to support data analytics and business decision making.

Key Responsibilities:

• Design, build and maintain scalable data pipelines and transformation processes using Microsoft Fabric components including Data Factory, OneLake, Dataflows and Notebooks.
• Develop and manage data models and analytics solutions.
• Develop ETL processes and integrate data from diverse sources (on-cloud and on-premise) into centralised and governed environments using Fabrics capabilities.
• Ensure data quality, integrity, consistency, security, governance and compliance with industry standards (Uniclass & SFG20) across all storage layers.
• Monitor performance, optimise data models and cost efficiency of Fabric components.
• Collaborate with bi developers, data analysts and developers to support initiatives by understanding business requirements and translate them into technical data solutions.
• Troubleshoot and resolve data related issues.
• Implement best practises for data engineering including coding standards, testing and deployment.
• Prepare and maintain documentation of data architecture, workflows and configurations.

Requirements:

Proven experience as a data engineer or in a similar role.

Strong proficiency in programming languages such as Python, TSQL, Power Query and KQL.

Strong understanding of Data Factory, data pipelines, Notebooks and Lakehouses within Fabric.

Familiarity with Microsoft security and governance frameworks, including Purview and RBAC.

Excellent analytical and problem solving skills.

Exposure to CI/CD practices and tools within Azure DevOps.

Desirable

• An understanding of the lifecycle of machine learning models.
• Experience with Azure Machine Learning or other cloud-based ML platforms for deploying and managing models.