Lead the architecture and design of scalable data platforms and large-scale distributed data processing systems.
Build and evolve robust data pipelines for ingestion, processing, enrichment, and transformation of structured and unstructured datasets.
Design and implement high-performance data models that support machine learning pipelines, analytics, and customer-facing applications.
Establish and drive best practices for data quality, observability, governance, and reliability across the data platform.
Partner closely with data science teams to support feature engineering, model training pipelines, and production ML systems.
Evaluate and introduce modern data platform technologies to improve scalability, performance, and developer productivity.
Drive adoption of LLMs and AI-powered tooling to improve data engineering productivity and apply AI-driven approaches for data discovery, pipeline automation, metadata enrichment, and data quality.
Mentor engineers and provide technical leadership across the data platform team.
Requirements
Bachelor's degree or higher in Computer Science, Engineering or related field with 13+ years of experience in data engineering with a strong focus on designing and building scalable data platforms and products.
Deep expertise in data modelling, distributed data processing, and large-scale ETL/ELT systems to build data lakes, data warehouses and modern lakehouse architectures.
Strong experience with distributed data technologies (Spark, Kafka, Hadoop, or similar frameworks) and programming expertise in Python and SQL.
Experience with cloud, such as AWS, Azure or GCP and related services (S3 Redshift, BigQuery, Dataflow).
Proven expertise in data quality checks to ensure data accuracy, completeness, consistency, and timeliness.
Hands-on experience using LLMs, AI-assisted development tools, or AI-driven workflows to improve engineering productivity and automate development processes.
Excellent problem-solving in a fast-paced, collaborative environment, coupled with strong communication for effective interaction with tech and non-tech stakeholders.
This job was posted by Sanoop Kannoli from Enlyft.
