We are seeking a strategic and technically accomplished Associate Director of Data Engineering to drive our Azure-based data platform initiatives. This role requires a leader who can design innovative cloud-native solutions while managing end-to-end delivery of complex data engineering programs across the organization.
Key Responsibilities
Solutioning & Architecture:
- Design comprehensive data engineering solutions utilizing Azure cloud services (Azure Data Lake Storage, Azure Synapse Analytics, Azure Data Factory, Azure Databricks, Event Hubs)
- Architect modern data platforms supporting batch and real-time data processing at enterprise scale
- Define data integration patterns, medallion architectures, and lakehouse implementations
- Establish data mesh principles and domain-driven data architectures where appropriate
- Create technical roadmaps and reference architectures aligned with business strategy and cloud best practices
- Design security, governance, and compliance frameworks for cloud data platforms
Delivery Management:
- Oversee multiple strategic data engineering programs from conception through production deployment
- Manage project portfolios including scope, schedule, budget, and resource allocation
- Drive stakeholder engagement across business units, ensuring solutions meet functional and non-functional requirements
- Implement agile delivery methodologies and establish metrics for tracking progress and success
- Facilitate decision-making processes and resolve conflicts across technical and business teams
- Manage vendor relationships and third-party integrations as needed
Team Leadership:
- Lead, mentor, and develop a diverse team of data engineers, architects, and analytics professionals
- Establish technical standards, coding guidelines, and best practices for the data engineering practice
- Promote a culture of innovation, accountability, and continuous improvement
- Conduct talent assessments, succession planning, and skill development initiatives
- Champion knowledge sharing through documentation, training sessions, and technical forums
Technical Delivery:
- Provide hands-on technical leadership in designing and implementing complex data pipelines and workflows
- Optimize data platform performance, cost efficiency, and reliability
- Implement observability frameworks including monitoring, alerting, and logging solutions
- Ensure data quality, lineage tracking, and metadata management capabilities
- Drive automation through CI/CD pipelines using Azure DevOps or GitHub Actions
Required Qualifications
- Bachelor's or Master's degree in Computer Science, Information Systems, Engineering, or related field
- 15+ years of progressive experience in data engineering with minimum 3 years in senior leadership roles
- Deep expertise in Azure cloud platform and services (Azure Synapse, Data Factory, Databricks, Data Lake Storage Gen2, Event Hubs, Stream Analytics)
- Strong programming skills in Python, SQL, and Spark (PySpark/Scala)
- Proven experience architecting and delivering enterprise-grade data platforms on Azure
- Strong understanding of data modeling, data warehousing, and modern data architecture patterns
- Experience with orchestration tools (Azure Data Factory, Apache Airflow)
- Solid grasp of DevOps practices and infrastructure-as-code (ARM templates, Bicep, Terraform)
- Demonstrated success managing complex technical programs and cross-functional teams
- Excellent communication skills with ability to engage both technical and executive audiences
Preferred Qualifications
- Azure certifications (Data Engineer Associate, Solutions Architect Expert, or similar)
- Experience with Power BI for analytics and reporting integration
- Knowledge of Azure Purview for data governance and catalog management
- Familiarity with Cosmos DB, Azure SQL Database, and other Azure data services
- Experience implementing data mesh or data fabric architectures
- Understanding of machine learning operations and integration with Azure ML
- Background in financial services, healthcare, retail, or other regulated industries
- Project management certifications (PMP, Prince2, SAFe)
- Experience with multi-cloud or hybrid cloud architectures
