Job DescriptionData Platform Architecture & Strategy
- Design and maintain the overall data platform architecture integrating Snowflake, AtScale, and dbt components, GBQ. Mandatory Google Platform
- Establish data quality frameworks and monitoring systems
- Define platform scalability and performance optimization strategies
- Design and implement Snowflake warehouse configurations and resource management policies
- Manage user access controls, roles, and security policies within Snowflake
- Optimize query performance through warehouse sizing, clustering, and partitioning strategies
- Implement data sharing and collaboration features across business units
- Monitor and control Snowflake costs through usage optimization and resource scheduling
- Design and maintain data pipeline architectures for ingestion and transformation
- Establish connection protocols between AtScale and various BI tools (Tableau, Power BI, etc.)
- Optimize query performance through aggregate tables and caching strategies
- Manage AtScale cluster configurations and scaling policies
- Implement security models that align with organizational data access requirements
dbt Development & Operations
- Manage dbt deployment pipelines and environment promotion processes
- Design and maintain dbt project structures and development workflows
- Create and enforce dbt coding standards and documentation requirements
- Implement dbt testing frameworks for data quality validation
- Design modular dbt models that support reusability and maintainability
- Establish version control and collaboration practices for dbt development teams
Performance & Cost Optimization
- Monitor and optimize compute resource utilization across all platforms
- Implement cost allocation and chargeback models for different business units
- Analyze query patterns and usage metrics to identify optimization opportunities
- Design and implement automated scaling policies based on workload demands
- Establish SLA monitoring and reporting for platform availability and performance
Integration & Data Flow Management
- Design end-to-end data pipelines connecting source systems to Snowflake via dbt transformations
- Create monitoring and alerting systems for data pipeline health and performance
- Implement data lineage tracking across all three platforms
- Establish disaster recovery and backup procedures for the integrated platform
- Design API integrations and data exchange protocols between platforms
Team Leadership & Collaboration
- Mentor and train data engineers and analysts on platform best practices
- Collaborate with data science teams to support ML/AI workload requirements
- Coordinate with DevOps teams on infrastructure automation and deployment practices
- Lead architectural review sessions and technical decision-making processes
Security & Compliance
- Implement data encryption, masking, and anonymization policies
- Ensure compliance with regulatory requirements (GDPR, HIPAA, SOX, etc.)
- Design and maintain audit trails and data access logging
- Coordinate security assessments and vulnerability management
- Establish data classification and handling procedures across all platforms
