Key Responsibilities
Data Pipeline Development (Microsoft Fabric)
- Design, develop, and deploy scalable data pipelines using Microsoft Fabric components such as OneLake, Data Factory, and Apache Spark
- Ensure efficient, secure, and reliable data movement across enterprise systems
- Build batch and near real-time data processing workflows
ETL Architecture & Data Engineering Design
- Architect ETL workflows optimized for Microsoft Fabric's unified data platform
- Streamline ingestion, transformation, and storage processes across structured and unstructured data
- Develop reusable and scalable data engineering frameworks
Data Integration & Platform Engineering
- Integrate diverse data sources into Microsoft Fabric OneLake ecosystem
- Use SQL, Python, Scala, and R for data transformation and processing
- Support enterprise-scale analytics through integration with Synapse and related services
OneLake & Synapse Analytics Implementation
- Leverage OneLake as a centralized enterprise data lake for analytics and AI workloads
- Enable data warehousing and advanced analytics using Synapse integration
- Support Power BI and downstream reporting systems
Performance Optimization & Reliability
- Monitor, troubleshoot, and optimize data pipelines for performance and scalability
- Ensure high availability, low latency, and minimal downtime in production systems
- Improve query performance and resource utilization across Fabric workloads
Data Governance, Security & Compliance
- Implement data governance frameworks including lineage, privacy, and access control
- Ensure compliance with enterprise data security standards
- Maintain auditability and data quality across the platform
Leadership & Team Mentoring
- Lead and mentor data engineering teams
- Oversee Fabric workspace design, code reviews, and technical best practices
- Drive adoption of Microsoft Fabric capabilities within the organization
Automation, Orchestration & DevOps
- Automate workflows using Fabric Data Factory, Azure DevOps, and Airflow
- Implement CI/CD pipelines for data engineering solutions
- Ensure smooth orchestration and operational efficiency of data systems
Innovation & Continuous Improvement
- Stay updated with Microsoft Fabric advancements including Real-Time Analytics, Data Activator, and AI integrations
- Identify and implement innovative solutions for enterprise data challenges
- Contribute to engineering best practices and platform modernization initiatives
