
Search by job, company or skills
ROLE OVERVIEW
We are seeking a highly skilled and motivated Data Engineer to join our team and play a pivotal role in building the Whole-of-Government (WOG) data platform. You will design, develop, and maintain robust data solutions that power data generation, collection, and processing across multiple government agencies.
As a subject matter expert, you will lead and collaborate across teams, drive key architectural decisions, and ensure that data solutions are scalable, reliable, and fit for purpose. This is a senior, hands-on role with significant cross-team impact.
KEY EXPECTATIONS
. Build and operate the core Whole-of-Government (WOG) data platform, ensuring high availability, scalability, and performance.
. Support government agencies in integrating seamlessly with the platform, providing technical guidance and hands-on assistance.
. Develop data connectors and pipelines to onboard agency datasets efficiently, minimising time-to-value for new integrations.
. Act as the primary technical authority for data engineering decisions within the team and across stakeholder groups.
. Drive continuous improvement of data infrastructure, tooling, and processes to meet evolving platform demands.
RESPONSIBILITIES
1. Data Platform & Architecture
. Design, develop, and maintain scalable data pipelines and ETL/ELT processes for data migration and deployment across systems.
. Build and operate the core WOG data platform, ensuring reliability, security, and performance at scale.
. Architect data ingestion, transformation, and storage solutions that support diverse agency requirements.
2. Agency Integration & Onboarding
. Develop reusable data connectors and pipeline templates to accelerate onboarding of agency datasets.
. Collaborate with agency teams to understand data formats, governance requirements, and integration constraints.
. Provide technical consultation and support to agencies throughout their integration lifecycle.
4. Data Quality & Governance
. Implement and enforce data quality checks, monitoring, and alerting across all data flows.
. Establish and uphold data access policies, lineage tracking, and audit mechanisms.
. Ensure compliance with data governance standards and security requirements.
3. Team Leadership & Collaboration
. Manage and mentor the engineering team, fostering a culture of technical excellence and continuous learning.
. Engage with multiple teams and senior stakeholders to align on platform direction and priorities.
. Contribute to key architectural and engineering decisions with clear, evidence-based recommendations.
. Provide solutions to complex problems for both the immediate team and cross-functional groups.
WHAT WE'RE LOOKING FOR
. Extensive experience designing and building large-scale data platforms and infrastructure.
. Proven track record in delivering ETL/ELT pipelines, data connectors, and integration solutions.
. Strong understanding of data quality, governance, lineage, and access control principles.
. Experience collaborating with cross-functional stakeholders and leading engineering teams.
. Excellent communication skills, with the ability to translate technical concepts for non-technical audiences.
. Passion for building platforms that create lasting impact across public sector organisations.
Job ID: 147085127
Skills:
.NET, data monitoring , Java, Data Quality, Scala, Data Governance, Python, Data Processing, data ingestion
Skills:
data engineering , Data Modelling, Systems Integration, Data Integration, Api Development, Automation Workflows, Full-Stack Development, ETL Pipelines, SaaS Platform Integrations
Skills:
Java, Performance Tuning, Node.js, REST, Python, reliability patterns, scalable storage systems, time-series databases, Go, batching, GRPC, data ingestion pipelines, data lakes, Relational Databases, backpressure, data partitioning, cloud-based data platforms, observability

Skills:
Sql, Databricks, ELT, Apache Flink, Azure Storage, Microservices, Data Warehousing, Pyspark, Kafka, Spark Streaming, Business Intelligence Development, Etl, Kubernetes, Python, Docker, Scala, Data Modelling, Event Grid, Data Lakes, Delta Live Tables, Event Hub, dbt, Helm Charts, Cloud Data Technologies
Skills:
Sql, Data Modeling, Java, Hadoop, Tableau, Power Bi, Hive, Data Warehousing, Python, Spark, Looker, ETL design patterns, cloud platforms
We don’t charge any money for job offers