Job Title: Sr. Data Engineer
Work Experience: 4-6 Years (Senior)
Work Experience: 2 to 3+ years (Mid-Level)
Work Experience 0 – 1+ years (Junior)
Work Location: India or Malaysia
Contract Tenure: Permanent after probationary period
Key Responsibilities
- Design, develop, and maintain end-to-end data pipelines using Microsoft Fabric
- Participate in data modelling activities using star and snowflake schemas
- Implement and manage Medallion Architecture (Bronze, Silver, Gold layers) to ensure scalable and governed data processing
- Apply data cleansing in silver layers and business-ready gold-datasets for optimizing analytics
- Implement quality checks, validation rules, and reconciliation processes
- Designed and delivered dbt‑driven Medallion Architecture (reusable dbt models, macros, and tests, enforcing data quality, scalability, and transformation standards).
- Implemented CI/CD deployment pipelines on Microsoft Fabric, enabling automated build, test, and deployment of Fabric artifacts (Lakehouse, notebooks, pipelines, semantic models) across environments
- Ingest data from multiple source systems (databases, files, APIs, cloud storage) using Fabric Data Factory
- Build and manage Fabric Lakehouse solutions leveraging OneLake
- Develop data transformation logic using PySpark and SQL for large datasets
- Monitor pipeline execution, handle failures, and implement robust error handling
- Work closely with business stakeholders to translate data requirements into technical solutions
- Ensure data security, access control, and compliance with governance standards
- Document data flows, transformations, and architecture decisions
- Perform root cause analysis for data issues and implement preventive fixes
- Support production deployments and post-release issue resolution
- Continuously evaluate and adopt best practices for Microsoft Fabric and analytics engineering
- Collaborate with BI teams to support Power BI Direct Lake and semantic models
Required Skills & Experience
- 4 to 6 years of hands-on experience in Data Engineering or Analytics Engineering
- Proven experience working on modern cloud-based data platforms
- Strong hands-on experience with Microsoft Fabric, LakeHouse mindset.
- Solid understanding and practical implementation of Medallion Architecture
- Proficiency in Spark, PySpark
- Strong SQL knowledge for data transformation and analysis
- Strong in CI/CD automatic deployment pipelines.
- Experience building and managing ETL / ELT pipelines
- Microsoft Fabric certifications (DP - 700) (Azure)
Data Engineering Skills
- Data modelling for analytics (star and snowflake schemas)
- Experience with batch and incremental processing
- Data quality, validation, and reconciliation techniques
- Performance optimization and troubleshooting
Good to Have
- Having experience to build dbt-driven end-to-end pipelines
- Power BI experience (Direct Lake preferred)
