We are seeking a highly skilled and experienced Senior Data Engineer with expertise in the Azure Cloud Platform, including Databricks, SQL, Azure Data Factory, Power BI, and SSIS. The ideal candidate will be responsible for designing, developing, and supporting robust data pipelines and solutions that enable our clients to achieve their business goals. This role requires a deep understanding of data engineering principles, cloud technologies, and a strong willingness to work on both support and development projects.
Key Responsibilities
Data Pipeline Development:
Design, develop, and maintain scalable data pipelines using Azure Data Factory and Databricks.
Optimize data workflows and ETL processes to ensure data quality and integrity.
Database Management
Develop and manage complex SQL queries and stored procedures.
Ensure efficient data storage and retrieval practices within the Azure SQL Database.
Business Intelligence And Reporting
Create and maintain interactive dashboards and reports using Power BI.
Collaborate with business stakeholders to gather requirements and translate them into technical solutions.
Integration Services
Design and implement ETL processes using SQL Server Integration Services (SSIS).
Manage and troubleshoot SSIS packages to ensure seamless data integration.
Support And Troubleshooting
Provide technical support for data-related issues and perform root cause analysis.
Collaborate with cross-functional teams to resolve data-related problems and optimize performance.
Project Management
Lead data engineering projects from inception to completion, ensuring timely delivery and adherence to quality standards.
Document technical designs, processes, and procedures for both development and support activities.
Continuous Improvement
Stay updated with the latest advancements in Azure cloud technologies and data engineering practices.
Propose and implement improvements to existing data processes and systems.
Qualifications
Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.
5+ years of experience in data engineering with a strong focus on the Azure cloud platform.
Proficiency in Databricks, SQL, Azure Data Factory, Power BI, and SSIS.
Solid understanding of data modeling, ETL processes, and data warehousing concepts.
Experience with Python, Spark, or other programming languages is a plus.
Strong analytical and problem-solving skills.
Excellent communication and collaboration abilities.
Ability to manage multiple tasks and projects simultaneously.
Willingness to work in both support and development roles.
Preferred Skills
Experience with other Azure services such as Azure Synapse Analytics, Azure Data Lake Storage, and Azure DevOps.
Familiarity with data governance and security best practices.
Knowledge of Agile methodologies and project management tools.