Data Engineer

Project Role : Data Engineer

Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems.

Must have skills : Data Engineering

Good to have skills : NA

Minimum 7.5 Year(s) Of Experience Is Required

Educational Qualification : 15 years full time education

Data Engineering Lead – Azure Data Platform, Lakehouse

Experience: 12+ years overall, including 5+ years leading data engineering teams or platform initiatives

Location: India

Role Summary

A hands-on technical lead who still builds: personally designing and writing core ETL/ELT pipelines and graph data models — both property-graph and RDF/semantic-graph — for the highest-priority integrations, while setting the lakehouse, medallion, data mesh, and catalog standards the rest of the team builds against. Owns the enterprise data platform architecture for an organization integrating data from many source systems, and leads by pairing and debugging alongside the team, not only through design sign-off.

Key Responsibilities

Personally design, build, and maintain ETL/ELT pipelines and transformation logic for the highest-priority and most complex data integrations — extract, transform (PySpark/Spark SQL), and load into bronze/silver/gold lakehouse tables

Write and review transformation logic for incremental loads, change data capture (CDC), deduplication, slowly changing dimensions (SCD), and schema evolution across the team's pipelines

Write and review Gremlin graph models and traversal queries in Azure Cosmos DB for Apache Gremlin for knowledge-graph and relationship-heavy use cases

Write and review SPARQL queries and Turtle (.ttl) ontology/data files for RDF-based knowledge graph and semantic layer work, using a triplestore such as Graphwise GraphDB

Set enterprise data architecture standards — lakehouse design, medallion layering conventions, incremental-load/CDC patterns, and data-store selection (relational, NoSQL, graph) — by building the reference pipelines the team follows

Own platform strategy and hands-on build-out for a modern Azure data platform (e.g., Microsoft Fabric) — unified lake storage, Data Factory/Dataflow Gen2 pipelines, Spark-based data engineering, and real-time data flows

Own the enterprise data catalog and governance strategy (e.g., Microsoft Purview) — lineage, classification, glossary, and stewardship model

Lead, mentor, and grow a team of data engineers through pairing, code review, and hands-on pipeline debugging — not only design sign-off

Partner with enterprise/solution architects, AI engineering leads, and business stakeholders to translate data strategy into a delivery roadmap

Own data security, access control, and compliance posture — RBAC, classification/sensitivity labeling, and audit — across the data platform

Required Skills & Experience

12+ years in data engineering/data platform roles, including 5+ years leading data engineering teams — while remaining hands-on and writing production code

Deep, current hands-on experience building ETL/ELT pipelines — incremental loads, CDC, deduplication, SCD, and schema evolution — on a modern Azure data platform (e.g., Microsoft Fabric, or Azure Synapse/Databricks-equivalent)

Strong Python and Spark (PySpark) for transformation development strong SQL for analytical and transactional workloads

Proven track record building and operating medallion (bronze/silver/gold) lakehouse pipelines at enterprise scale, not only designing them on paper

Hands-on experience writing graph data models and Gremlin queries in Azure Cosmos DB for Apache Gremlin

Hands-on experience with RDF/semantic graph technologies — writing SPARQL queries and authoring Turtle (.ttl) files, using a triplestore such as Graphwise GraphDB (or equivalent, e.g., Amazon Neptune, Stardog)

Experience designing and rolling out data mesh operating models — domain ownership, data-as-a-product, self-serve platforms, and federated governance

Experience owning enterprise data catalog / governance tooling (e.g., Microsoft Purview) — data map, unified catalog, lineage, stewardship

Track record integrating and rationalizing data from many heterogeneous enterprise source systems (ERP, CRM, SaaS, APIs, legacy databases)

Experience mentoring engineers through hands-on pairing and code review, and presenting data architecture/strategy to senior stakeholders

Preferred

Exposure to the broader semantic web stack — RDF/RDFS, OWL, SHACL, SKOS — and ontology-driven knowledge graph / semantic layer architecture

Experience with pipeline orchestration tools (e.g., Apache Airflow, or Fabric's Airflow-based orchestration)

Experience across multiple major cloud data platforms (AWS, Azure, and GCP) — data lake, warehouse, and serverless/ETL services

Familiarity with vector stores and data preparation patterns for RAG/agentic AI consumption

Experience with real-time/streaming architectures (e.g., Eventstream, KQL, or equivalent)

Exposure to regulated-industry data governance (e.g., life sciences, healthcare, financial services)