What is this role
The Senior Principal Data Platform Engineer is a strategic technical leader responsible for shaping the enterprise data platform vision and driving modernization initiatives. This role leads complex, cross-functional projects, including the migration from legacy Microsoft BI technologies to modern cloud-native platforms such as Databricks, while ensuring governance, compliance, and operational excellence. The position demands deep technical expertise, strong leadership, and a passion for building scalable, secure, and future-ready data solutions.
What will I be doing
Platform Modernization & Migration
- Architect and lead large-scale migrations from MS BI stack (SSIS, SSAS, Power BI, SQL Server) to Databricks and other modern data platforms.
- Develop migration strategies that minimize downtime, ensure data integrity, and meet compliance requirements.
- Evaluate and implement tools and frameworks for seamless data movement and transformation.
Data Architecture & Engineering
- Design and maintain scalable, resilient data pipelines and ETL/ELT processes using Spark-based platforms (Databricks, Microsoft Fabric).
- Define reusable patterns and frameworks for efficient data transformation and integration.
- Own critical components of the enterprise data architecture, ensuring alignment with business objectives and scalability.
Governance, Security & Compliance
- Establish and enforce robust data governance practices, including data quality, lineage, privacy, and security.
- Ensure compliance with regulatory standards (e.g., GDPR, HIPAA) and internal policies.
- Advocate for best practices in data stewardship across teams.
DevOps, Source Control & CI/CD
- Implement and champion best practices for source control (GitHub, Azure DevOps) and CI/CD pipelines in data engineering.
- Integrate automated testing, code reviews, and deployment strategies to ensure reliability and maintainability.
- Drive adoption of Infrastructure-as-Code and automated provisioning for data environments.
Leadership & Collaboration
- Provide technical leadership and mentorship to data engineers and cross-functional teams.
- Partner with product managers, analysts, and engineering teams to deliver solutions that solve real business problems.
- Act as an evangelist for modern data strategies and continuous improvement.
Innovation & Continuous Learning
- Stay ahead of emerging technologies in distributed computing, data architecture, and cloud platforms.
- Lead evaluations and adoption of new tools and methodologies to keep the data platform future ready.
We'd love to hear from you if:
- Experience: 15+ years in data engineering or related fields, with at least 5 years in a senior/principal role.
- Migration Expertise: Proven track record of migrating legacy data warehouses and BI solutions to Databricks or similar platforms.
- Technical Skills:
- Strong SQL and dimensional modeling (Kimball, star/snowflake schemas).
- Expert-level MS SQL Server experience (including performance tuning, query optimization, and schema design.
- Hands-on experience with Spark, Databricks, and cloud data services.
- Familiarity with Microsoft BI stack (SSIS, SSAS, Power BI).
- Governance & Compliance: Deep understanding of data governance frameworks and regulatory compliance.
- DevOps: Expertise in source control, CI/CD, and automated testing in data engineering contexts.
- Leadership: Exceptional communication, problem-solving, and mentoring skills.
- Preferred: Experience supporting Data Science and ML initiatives.
