Director - Data Engineering

Role Summary

• Own the Databricks Lakehouse end-to-end: ingestion, Medallion (Bronze/Silver/Gold) modeling, Unity Catalog governance, Lakeflow pipelines, and performance.

• Design and build robust, scalable pipelines that bring together media and marketing data from many platforms into governed, analytics-ready datasets.

• Drive adoption of AI-assisted data development across the team using Databricks Genie, Copilot, and Claude Code to accelerate pipeline build, transformation, and debugging without sacrificing rigor.

• Establish and enforce data taxonomies, schema standards, data quality checks, and a single source of truth for core media data.

• Partner with visualization, data science, and Ad Ops teams to source data correctly, validate it, and resolve discrepancies between platform truth and downstream reporting.

• Contribute to the productization of the data platform—moving from bespoke, client-by-client pipelines toward reusable, standardized, semi-automated data products.

• Manage capacity, utilization, and delivery health for the DGS engineering team, coordinating closely with onshore leadership on staffing and priorities.

Core Skills & Requirements

Databricks & Data Engineering

• Deep, hands-on Databricks expertise: Spark, Delta Lake, Unity Catalog, Lakeflow/Delta Live Tables, Medallion architecture, and performance tuning.

• Expert-level SQL and strong Python for data engineering; production experience building and operating pipelines at scale.

• Solid grasp of data modeling, orchestration, data quality, and governance—building for reuse and maintainability, not one-off delivery.

• Experience with Azure and lightweight ETL/data-prep tools such as Trifacta or dbt is a strong plus.

AI-Assisted Data Development

• Hands-on experience developing data solutions with AI tooling such as Databricks Genie, GitHub/Databricks Copilot, Claude Code, or comparable AI coding assistants.

• Able to use AI assistants to accelerate pipeline development, transformation logic, and debugging while maintaining correctness, governance, and quality.

• A builder's mindset, using AI as a force multiplier rather than a novelty.