Data Engineer

Summary

A hands-on Full Stack Data Engineer responsible for designing, building, and optimizing scalable Microsoft Fabric-based data and analytics solutions. The role requires expertise across data engineering, cloud integration, AI-assisted development, and lightweight application integration, with a focus on rapid delivery, reliability, scalability, and business impact.

Full Stack Data Engineer (Microsoft Fabric) – 5+ Years experience

Location- Chennai, Pune

Experience 5+ Years

Overview

We are seeking a highly motivated and hands-on Full Stack Data Engineer with strong experience in Microsoft Fabric and modern Azure-based data platforms. The ideal candidate should be capable of working across the full stack of data engineering — from ingestion and transformation to Gold-layer curation, analytics enablement, API integration, and AI-assisted application workflows.

This role requires engineers who can work independently, leverage AI-assisted development for rapid delivery, and collaborate across data, cloud, and lightweight application layers. Exposure to MCP (Model Context Protocol), ReactJS integration, and modern AI-enabled engineering practices is highly preferred.

1. Fabric Ecosystem & Full Stack Data Engineering

• Work with organizational OneLake structures, creating and managing shortcuts for efficient enterprise-scale data access

• Design and maintain scalable Lakehouse solutions using Medallion Architecture principles (Bronze, Silver, Gold)

• Build and optimize Delta Lake tables for reporting, analytics, and AI workloads

• Develop and manage pipelines using Fabric Data Factory, Notebooks, and Spark workloads

• Build ingestion and transformation workflows supporting structured and semi-structured data

• Implement orchestration, scheduling, monitoring, and recovery mechanisms for enterprise data pipelines

• Implement dimensional models (Star Schema/Snowflake Schema) to support BI, reporting, and semantic layer requirements

• Build curated Gold-layer datasets for downstream analytics and AI consumption

• Support integration with Power BI semantic models and reporting platforms

2. Azure, Integration & Full Stack Development

• Develop batch and incremental pipelines from ADLS Gen2, APIs, Azure SQL, Blob Storage, and external systems

• Support ETL/ELT orchestration using Fabric Pipelines and Azure Data Factory

• Integrate Fabric-based data platforms with APIs, AI services, and enterprise applications

• Support MCP (Model Context Protocol) integration and AI-enabled workflows where required

• Collaborate on AI-assisted development and rapid prototyping initiatives

• Work with lightweight ReactJS-based applications and frontend integrations

• Support development of internal dashboards, data-driven applications, and AI-enabled user experiences

• Support automation using Azure Functions, Logic Apps, and event-driven workflows

• Work with Git, CI/CD pipelines, Azure DevOps, and deployment automation processes

3. Data Processing, Optimization & AI-Assisted Development

• Develop and optimize PySpark notebooks for transformation, cleansing, and enrichment

• Build efficient SQL queries, views, and stored procedures in Fabric Warehouse / Azure SQL

• Implement optimization techniques including partitioning, caching, and query tuning

• Monitor pipeline performance, troubleshoot failures, and improve system reliability

• Implement logging, alerting, and operational best practices

• Utilize AI-assisted development tools such as GitHub Copilot and modern AI coding assistants

• Rapidly prototype and deliver scalable engineering solutions with minimal guidance

4. Governance, Security & Collaboration

• Implement RBAC and secure data access across Fabric workspaces and Azure environments

• Apply data quality validations and governance best practices

• Support metadata management and lineage using Microsoft Purview

• Collaborate with Data Architects, Analysts, BI Developers, Product Teams, and Business Stakeholders

• Translate business requirements into scalable data and application solutions

• Participate in Agile delivery processes, code reviews, and pull request workflows

Must-Have Skills

• Microsoft Fabric (Fabric Data Factory, OneLake, Lakehouse, Spark Notebooks)

• Azure Data Services: ADLS Gen2, Azure SQL, Blob Storage

• Strong SQL skills (joins, aggregations, optimization)

• Python / PySpark for data transformation

• ETL/ELT pipeline development

• Medallion Architecture (Bronze/Silver/Gold)

• Exposure to REST APIs and backend integrations

• Basic to intermediate ReactJS knowledge

• Understanding of AI-assisted development workflows

• Git, Azure DevOps, CI/CD

Good to Have

• MCP (Model Context Protocol) exposure

• Azure OpenAI / AI integration concepts

• Azure Functions / Logic Apps

• Event Hubs / streaming concepts

• Microsoft Purview

• Cosmos DB

• Power BI semantic models

Experience

• 5+ years of experience in Data Engineering

• Hands-on experience with Microsoft Fabric preferred

• Strong Azure Data Engineering background with willingness to work across full-stack and AI-enabled engineering workflows

Key Competencies

• Strong problem-solving and debugging skills

• Ability to work independently with minimal guidance

• Strong ownership mindset and delivery focus

• Good understanding of ETL/ELT and cloud-native data platforms

• Ability to collaborate across engineering, analytics, and application teams

• Adaptability in fast-paced AI-enabled development environments

Certifications (Preferred)

• DP-203: Azure Data Engineer Associate

• DP-700: Fabric Data Engineer Associate