Key Responsibilities:
Data Engineering & Pipeline Development
- Design, develop, and maintain scalable and reliable ETL/ELT pipelines.
- Build and optimize data ingestion and transformation workflows using SSIS, Azure Data Factory, and Azure Databricks.
- Ensure efficient movement and processing of large-scale structured and unstructured data.
Data Warehouse Architecture
- Design and implement enterprise data warehouse and data mart solutions.
- Apply Kimball methodology and dimensional data modeling techniques.
- Build scalable database structures including partitioning, indexing, compression, and encryption.
Cloud & Big Data Solutions
- Support cloud migration initiatives for data platforms (Azure-focused).
- Work with Azure Synapse, Databricks, and other MPP databases such as Snowflake.
- Integrate big data technologies including Spark and NoSQL databases.
Database Development & Optimization
- Develop and optimize SQL Server databases using advanced T-SQL.
- Perform query tuning, performance optimization, and troubleshooting.
- Manage large-scale database systems with high availability and scalability requirements.
Data Quality & Governance
- Implement data quality, profiling, metadata management, and Master Data Management (MDM) practices.
- Ensure data integrity, consistency, and reliability across systems.
- Enforce data governance standards and documentation best practices.
BI & Reporting Support
- Collaborate with BI teams to support dashboards, reports, and self-service analytics solutions.
- Identify and resolve data-related issues impacting reporting and analytics.
- Assist in defining business metrics and reporting requirements.
Performance Monitoring & Optimization
- Monitor data pipelines and BI solutions for performance and reliability.
- Implement automation for data workflows and operational processes.
- Continuously refine and enhance data processing systems.
Collaboration & Stakeholder Management
- Work closely with business teams to gather requirements for data and analytics solutions.
- Provide technical leadership and guidance to engineering teams.
- Participate in agile ceremonies, design reviews, and process improvements.
