Lead Data Engineer

we are looking for an experienced Data Engineer with strong expertise in Databricks, Microsoft Fabric, or Snowflake, along with Power BI. The candidate will be responsible for designing, developing, and optimizing scalable data pipelines, implementing modern data architectures, and delivering analytics solutions to support business decision-making.

Key Responsibilities

• Design, develop, and maintain ETL/ELT pipelines using Databricks, Microsoft Fabric, Azure Data Factory (ADF), or Snowflake.
• Build and optimize data workflows for performance, scalability, and cost efficiency across cloud environments.
• Develop data processing and transformation solutions using Python, PySpark, and SQL.
• Implement Bronze, Silver, and Gold layers using Medallion Architecture.
• Design and maintain data lakes, data warehouses, and lakehouse architectures.
• Develop Power BI dashboards and reports to deliver actionable business insights.
• Integrate data engineering solutions with Power BI and downstream analytics applications.
• Implement data quality checks and ensure data integrity, security, and reliability.
• Collaborate with cross-functional teams to define data requirements, governance standards, and engineering best practices.
• Automate workflows and support CI/CD deployments using Git and DevOps practices.
• Monitor, troubleshoot, and optimize data pipelines and dashboards.
• Support data migration projects from legacy platforms to modern data platforms.

Required Technical Skills

• Data Platforms: Hands-on experience with at least one of Databricks, Microsoft Fabric, or Azure Data Factory.
• Programming: Strong Python programming skills.
• Big Data Processing: Strong PySpark / Apache Spark experience.
• SQL: Strong SQL skills, including complex queries and performance optimization.
• Power BI: Hands-on experience with data modeling, DAX, dashboard development, and performance optimization.
• Data Integration: Experience developing scalable ETL/ELT pipelines.
• Data Architecture: Strong understanding of Medallion Architecture, Star Schema, dimensional modeling, and lakehouse concepts.
• Cloud Platforms: Experience with at least one major cloud platform: Azure, AWS, or GCP.
• Microsoft Fabric: Hands-on experience with Dataflow Gen2 for low-code data ingestion and transformation.
• Data Migration: Experience migrating data from legacy platforms, such as Informatica to Fabric, SSIS to ADF, or legacy systems to Databricks/Fabric.

Preferred Skills

• Experience with Delta Lake and modern cloud data architectures.
• Knowledge of Azure Synapse Analytics or Snowflake.
• Experience with Databricks platform optimization and its ecosystem.
• Familiarity with CI/CD, Git, DevOps, and Infrastructure as Code.
• Understanding of data governance, lineage, security, and compliance.
• Experience with streaming technologies such as Kafka, Azure Event Hubs, or Amazon Kinesis.
• Exposure to machine learning pipelines, advanced analytics, AI Ops, or GenAI data workloads.
• Databricks professional certification is an advantage but not mandatory.

Experience Requirements

• 5–10 years of relevant experience in Data Engineering.