Senior Data Engineer

About the Role

As a Senior Data Engineer, you'll be at the forefront of our data infrastructure, designing and building scalable data pipelines that transform raw information into valuable business intelligence. You'll lead technical initiatives, make architectural decisions, and collaborate with cross-functional teams to understand data needs and implement solutions that enable data-driven decision making across the organization.

Key Responsibilities

• Lead and design robust, scalable data pipelines and ETL processes using Python, PySpark, and other modern technologies
• Architect and implement complex data models and structures in various databases including MongoDB and SQL Server
• Build and optimize enterprise-level data workflows using Palantir / Azure Databricks platforms to ensure efficient data processing
• Drive collaboration with data scientists, analysts, and business stakeholders to understand data requirements and deliver appropriate solutions
• Establish data quality standards by implementing validation procedures, monitoring systems, and data governance practices
• Lead troubleshooting efforts for complex data issues while maintaining system performance
• Evaluate and recommend new data technologies to continuously improve our data infrastructure
• Create and maintain technical specifications, processes, and best practices for sharing knowledge
• Mentor and develop junior engineers and contribute to establishing engineering standards across the organization
• Act as a Data Engineering SME supporting multiple teams across various domains and driving engineering best practices across all teams.

About the Team

We are a team that believes in engineering excellence and that our leaders should also be engineers themselves. We build applications that are carefully designed, thoughtfully implemented by working together with product owners. Quality and stability are first-class deliverables in everything we do, and we lead by example by embedding high standards into our processes.

About You

You're an experienced problem solver who thrives in collaborative environments and approaches challenges with creativity and analytical thinking. You have a deep passion for data engineering and stay current with emerging technologies. You're a natural leader who can guide technical discussions and mentor team members.

We are looking for candidates who meet these minimum requirements:

• Bachelor's degree in computer science, Engineering, Information Technology, or a related field, with 12+ years of experience in data engineering or related roles.
• Advanced proficiency in Python, with extensive experience designing and building enterprise-grade data pipelines and ETL/ELT processes.
• Deep expertise in PySpark for large-scale batch and distributed data processing in production environments.
• Comprehensive experience with Palantir Foundry, Azure Databricks, or similar enterprise data platforms for data integration, workflow orchestration, data engineering, and analytics at scale.
• Strong experience designing scalable data models, database schemas, and analytical data structures using techniques such as dimensional modelling, star schema, data warehouse design, and modern lakehouse patterns.
• Proven experience implementing data quality frameworks, validation checks, monitoring mechanisms, metadata management, and data governance practices.
• Hands-on experience troubleshooting complex data issues, optimizing pipeline performance, improving reliability, and maintaining production-grade data systems.
• Strong experience partnering with data scientists, analysts, product owners, and business stakeholders to translate business and analytical requirements into scalable data solutions.
• Demonstrated ability to lead technical initiatives, make architectural decisions, mentor junior engineers, define engineering standards, and promote best practices across teams.
• Demonstrated Ability working as a Data Engineering SME across multiple teams, projects, or business domains, with the ability to influence engineering practices beyond a single delivery team.
• Ability to evaluate new data technologies and recommend improvements to enterprise data platforms, tools, and engineering practices.
• Strong documentation skills, including the ability to create and maintain technical specifications, process documents, standards, and reusable best practices.

These are additional nice-to-have requirements:

• Experience with cloud platforms such as AWS, Azure, or GCP and their data services at enterprise scale.
• Experience with real-time or near-real-time data processing frameworks such as Kafka, Spark Streaming, or similar.
• Experience with workflow orchestration tools and CI/CD practices for data engineering.
• Advanced knowledge of data visualization tools such as Tableau, Power BI, or similar.
• Familiarity with MLOps, AI/ML data pipelines, or data products supporting analytics and AI use cases.
• Experience with TypeScript for data manipulation and data visualization.