Data Engineer

Looking for data problems actually worth solving 

Most data engineering roles are pipeline maintenance in disguise. You inherit someone else's spaghetti, patch it quietly, and call it a day. At Ideas2IT, you'll build data infrastructure that actually powers things: AI platforms, large-scale cloud-native applications, products that have gone on to become independently funded startups.

What you'll actually do as a Data Engineer with us:

- Design and build robust data pipelines and architectures on AWS or Azure

- Develop and optimize ETL processes that handle large-scale structured and unstructured data from multiple sources

- Work with distributed processing tools like PySpark and Pandas to move and transform data at scale

- Use cloud services hands-on: Glue, Lakeformation, EMR for AWS Datafactory, Synapse, Datalake,  Fabric on Azure

- Model and warehouse data that feeds BI tools and supports real analytical decisions

- Write clean Python or Scala to automate, integrate, and build

- Collaborate with analytics and business teams to translate data needs into infrastructure that works

- Contribute to tech stack decisions: your opinion on architecture matters here, not just your execution.

Must-haves:

- Strong SQL and hands-on experience with relational databases: MySQL, PostgreSQL, or SQL Server

- Strong hands-on experience with Cloud data warehouses:  AWS Redshift, Snowflake, Azure Synapse Analytics- Experience in at least one cloud environment: AWS or Azure

- Proven experience building large-scale data pipelines and ETL processes

- Proficiency with distributed data processing: PySpark, Pandas, Polars

- Fluency in Python and/or Scala

- Experience with cloud data services (Glue, Lakeformation, EMR for AWS Datafactory, Synapse, Datalake,  Fabric on Azure)

- Ability to handle messy, large-scale data from internal and third-party sources and make it usable

Nice-to-haves:

-Experience with data visualization tools: Power BI or Tableau

-Familiarity with orchestration and workflow management tools 

-Familiarity with Data quality, testing and observability tools

-Familiarity with Docker and Kubernetes

Why Ideas2IT – and why now

We operate at the intersection of a product studio and a services firm. That means your work spans industries, scales, and problem types, and not the same pipeline in a slightly different color for three years running. We've built data infrastructure for companies like Meta, Bloomberg, and Medtronic. We've also launched six of our own products through IdeaLabs, several of which have become independently funded startups: Element5 (a $50M health-tech platform), LegacyLeap, IdeaRx, PipeCandy, and more.

The shift to AI-native development is changing what great data engineering looks like. The engineers who thrive next aren't just pipeline builders; they're the ones who understand what AI systems need from data, and build infrastructure that delivers it. You'll develop that fluency here, on real products, not in a sandbox.

What that looks like day-to-day:

-Autonomy to contribute to architecture and tech stack choices, not just implement them

-Access to technical PoCs if there's a technology you want to explore

-Internal Shark Tanks and Hackathons if you want to take an idea further

-A culture that prioritises capability over tenure

The hiring process – we mean business & move fast.

-Send us something real: your resume, GitHub, a short note. We're not fussy.

-Do a real-world challenge: a task relevant to the work, not a whiteboard puzzle.

-Talk to the people you'd work with: no HR screening theatre.

-Get a fast answer: yes or no, with a reason either way.

Ideas2IT is an equal opportunity employer. We hire for craft, curiosity, and drive – period.

Learn more at https://www.ideas2it.com/careers