Lead Software Engineer

The Data Fabric Developer is responsible for the development, enhancement, and support of modern data integration and warehousing solutions. This role focuses on migrating existing on-premises SQL and SSIS-based data environments to the Microsoft Fabric ecosystem. The candidate will design and build scalable data solutions that leverage OneLake for unified data storage, implement Medallion Architecture (Bronze, Silver, Gold layers) for robust data pipelines, and ensure high-performance data delivery using Parquet and Delta Parquet formats.

Key Responsibilities

• Development & Migration: Develop and maintain ADF pipelines, PySpark notebooks, and T-SQL solutions to migrate on-premises workloads to Microsoft Fabric.
• Architecture Implementation: Design and implement the Medallion Architecture within the Fabric Lakehouse environment to organize data into logical Bronze, Silver, and Gold tiers.
• Storage & Format Optimization: Manage data storage strategies using ADLS Gen2 and OneLake; optimize data ingestion and processing by leveraging Parquet files and Delta Parquet tables for performance and reliability.
• Warehouse Management: Develop, manage, and optimize data models within the Microsoft Fabric Warehouse to support enterprise reporting and analytics.
• Data Transformation: Implement complex data transformations, validation rules, and automated load processes.
• Testing & Quality Assurance: Perform unit testing, manage defect resolution, and support integration and User Acceptance Testing (UAT) cycles.
• Performance Tuning: Optimize SQL queries, ETL processes, and compute usage to ensure system scalability and cost-efficiency.
• Process Compliance: Follow established coding standards, documentation practices, and deployment procedures.
• Collaboration & Support: Work alongside architects and leads to interpret design specifications, resolve technical roadblocks, and provide production support for data environments