Lead Data Engineer HIP Integration

Lead Data Engineer — HIP Integration

A senior, hands-on data engineer who can own the end-to-end onboarding of CMS/AS into the Heathrow Insights Platform — choosing the right ingestion pattern, landing data through Bronze into the shared Silver unifying source, and making it consumable in Gold and setting up PowerBI reports.

Core technical profile

• Lakehouse engineering on Databricks/Unity Catalog — Delta tables, Lakeflow Jobs, notebook-based orchestration, medallion (Bronze/Silver/Gold) modelling.
• Breadth across the four supported integration patterns: direct/streaming (Event Hubs Kafka endpoint, Kafka, Zerobus push), Lakeflow Connect managed connectors (Salesforce, SQL Server, Azure SQL, SFTP, Snowflake federation), Open Sharing for cross-platform access, and custom orchestrated REST/GraphQL ingestion. Must be able to choose the pattern, not just implement one.
• Batch, micro-batch and near-real-time pipeline design, with the roadmap moving toward near-real-time lakehouse and data APIs.

Leadership and ways of working

• Lead a team of 2 data engineers working on CMS and AS
• Sets and enforces engineering standards: reuse, discoverability, observability, documentation, and paying down technical debt rather than creating it.
• Confident stakeholder engagement with data product owners across a broad initiative portfolio (baggage, airside ops, carbon, forecasting, commercial finance).