Tech Specialist / Senior Staff Engineer

Role Overview

We are looking for a hands-on Tech Specialist Software Engineer to lead the design and execution of core capabilities within our Enterprise Data Platform (EDP), spanning data ingestion, access, and governance.

This role is for someone who can operate at both architectural and implementation depth, translating complex, ambiguous data requirements into scalable, production-grade systems, while actively contributing to code, design, and platform evolution.

You will play a critical role in building a platform-first ecosystem that enables teams to onboard, process, and consume data reliably, securely, and efficiently.

Key Responsibilities

1. Architecture & System Design

• Translate complex business and data requirements into scalable, resilient platform architectures

• Design end-to-end data flows across ingestion (batch/stream), transformation, and consumption layers

• Define platform patterns for ingestion, data access (APIs, query layers), and governance (metadata, lineage, policy enforcement)

• Clearly articulate trade-offs and decisions across architecture choices (e.g., platform vs bespoke pipelines, batch vs streaming, UI vs code-first)

2. Hands-On Engineering & Execution

• Actively contribute to production-grade code across ingestion frameworks, orchestration, and platform services

• Build and evolve reusable platform components (connectors, SDKs, APIs, templates)

• Drive implementation of data ingestion pipelines (e.g., CDC, batch, streaming) into platforms like BigQuery

• Develop services enabling data access patterns (data APIs, query abstraction layers, secure access mechanisms)

• Implement governance capabilities including metadata capture, lineage, data quality, and policy enforcement

3. Platform & Developer Experience

• Build a platform-first experience that abstracts complexity for downstream teams

• Enable both low-code/no-code (UI) and code-first (SDK/API) approaches for data onboarding and processing

• Improve developer productivity through standardization, templates, and automation

4. Data Governance & Observability

• Design and implement data quality frameworks, validation pipelines, and monitoring

• Enable end-to-end lineage and impact analysis

• Establish observability for data products (freshness, reliability, usage, cost)

• Ensure secure and compliant data access through policy-driven controls

5. Engineering Excellence

• Write clean, maintainable, and scalable code following best practices (SOLID, DRY, design patterns)

• Lead code reviews and drive a culture of high-quality engineering

• Perform root cause analysis and implement long-term fixes, not just patches

• Contribute to technical documentation, design reviews, and knowledge sharing

6. Collaboration & Leadership

• Partner closely with Product, Data Engineering, and Platform teams

• Help shape technical roadmaps and influence platform strategy

• Mentor engineers and drive engineering best practices across teams

• Act as a technical DRI for critical platform initiatives

What We're Looking For

• Strong experience in building large-scale data platforms or distributed systems

• Deep expertise in data ingestion patterns (CDC, streaming, batch)

• Experience with data access layers, APIs, and query optimization

• Strong understanding of data governance (metadata, lineage, data quality, access control)

• Proven ability to design AND implement complex systems (not just high-level architecture)

Technology Stack

• Programming: Python, Java (or similar)

• Data & Cloud: GCP (BigQuery, Dataflow, Dataproc, Pub/Sub, Datastream preferred)

• Frameworks: Spring Boot, Google ADK (or similar AI/agent frameworks where applicable)

• CI/CD & Infra: Terraform, GitHub Actions

• Metadata & Governance: DataHub (or similar tools)

• Orchestration: Airflow / Astronomer

Nice to Have

• Experience with AI Agents / LLM-powered workflows for data automation or observability

• Exposure to multi-modal data processing (logs, sensor, image data)

• Experience building platform SDKs or developer tooling

Who You Are

You are a builder at heart. You care about writing high-quality code, but also about the systems and platforms that code enables. You take ownership end-to-end, from problem definition to production, and you are driven by impact.

You bring a product mindset to platform engineering, understanding that the ultimate goal is to enable other teams to move faster and more effectively. You value collaboration, enjoy mentoring others, and continuously look for ways to improve both the system and the team around you.