Azure Data Engineer/Architect

We are seeking a highly skilled and experiencedAzure Data Engineerto join our team. The ideal candidate will have a strong background in cloud-based data integration, data transformation, and analytics solutions, with a focus on Azure services. This role involves designing, implementing, and maintaining robust data pipelines and analytics frameworks to support business intelligence and advanced analytics initiatives.

Key Responsibilities:

1. Data Engineering and Development:

• Design and implement scalable and efficient ETL/ELT processes usingAzure Data FactoryandAzure Databricks (PySpark).
• Manage and optimize data storage and access inAzure Data Lake Storage.

1. Data Architecture:

• Develop and maintain aLakehouse architecturefor data analytics and reporting.
• Ensure best practices in data organization, partitioning, and access control in Azure Data Lake.
• Collaborate on the design and implementation ofMicrosoft Fabric architectureto enhance data accessibility and governance.

1. Data Analysis and Reporting:

• Create and optimize reports and dashboards usingPower BIto enable data-driven decision-making.
• Develop T-SQL scripts and stored procedures for data transformation and analysis.

1. Cloud Infrastructure and DevOps:

• Leverage knowledge ofAzure IaaS servicesfor data platform setup and maintenance.
• Implement and manage CI/CD pipelines usingAzure DevOpsto automate data pipeline deployments.

1. Technical Expertise and Collaboration:

• Provide strong technical guidance inSpark architectureand ensure best practices for PySpark code development.
• Collaborate with cross-functional teams to align on data strategies and solutions.
• Troubleshoot and resolve complex data engineering challenges efficiently.

Mandatory Skills:

• Proficiency inAzure Data Factory,Azure Data Lake Storage, andAzure Databricks (PySpark).
• Strong understanding ofETL and ELTprinciples.
• Deep knowledge ofLakehouse architectureand its implementation.
• Expertise inPySparkandSpark architecture.
• Solid understanding ofAzure Data Lake architectureand access control mechanisms
• Strong command ofT-SQLfor advanced querying and data manipulation..

Good to Have:

• Experience withPower BIfor visualization and reporting.
• Familiarity withMicrosoft Fabric architecture.
• Knowledge ofAzure IaaS services.
• Understanding ofCI/CD processesand tools likeAzure DevOps.

Qualifications:

• Bachelors degree in Computer Science, Information Technology, or a related field.
• 5+ years of experience in data engineering with a focus on Azure services.
• Proven expertise in designing and implementing data pipelines and architectures in cloud environments.
• Strong problem-solving skills and ability to work collaboratively in a team.

Preferred Certifications:

• Microsoft Certified: Azure Data Engineer Associate.
• Microsoft Certified: Azure Solutions Architect Expert.

Microsoft Certified: Power BI Data Analyst Associate