Data Engineering Manager

Core Responsibilities

• Lead and mentor teams of Big Data Developers, Data Engineers, and Platform Specialists to deliver scalable data solutions.
• Define and enforce standards for data quality, governance, performance, and security.
• Oversee the design and migration of data pipelines, orchestration frameworks, and ingestion flows to the NexusOne platform.
• Collaborate with Platform and Solution Architects to ensure alignment with the North Star architecture.
• Implement CI/CD practices, code reviews, and deployment automation for all data workloads.
• Develop reusable frameworks, patterns, and best practices to accelerate platform adoption.
• Drive continuous improvement through performance optimization, automation, and observability.
• Engage with stakeholders across engineering, architecture, and operations to align priorities and ensure on-time delivery.

Must-Have Core Skills

• Distributed Data Systems: Deep expertise in Apache Spark (core, SQL, streaming) and Airflow orchestration.
• Open-Source Data Stack: Hands-on with Iceberg, Trino/Presto, Hive, NiFi, and Kafka.
• Data Modeling & Transformation: Strong command of SQL, DBT, and schema-on-read design principles.
• Programming & Scripting: Proficiency in Python, Scala, and Java for data pipeline development.
• Infrastructure & Automation: Practical experience with Kubernetes, Terraform, Helm, GitLab CI/Jenkins, and Linux-based environments.
• Governance & Security: Familiarity with Ranger, DataHub, RBAC, and enterprise IAM/SSO integration.
• Data Quality & Testing: Working knowledge of Great Expectations or similar validation frameworks.
• Delivery Leadership: Proven ability to manage distributed engineering teams in Agile/DevOps settings.

Ideal Background

• 812 years of experience in data engineering or platform modernization, including 3+ years leading technical teams.
• Strong foundation in open-source and hybrid-cloud data ecosystems.
• Demonstrated success leading migration or modernization programs at enterprise scale.
• Excellent communication and stakeholder-management skills; capable of bridging executive vision and technical execution.