We are looking for a highly skilledLead Data Engineerwith expertise inAzure, Python, Databricks, and ETLto drive data engineering initiatives. The ideal candidate will have strong leadership experience, hands-on technical skills, and a deep understanding of cloud-based data solutions.
Key Responsibilities:
- Lead the design, development, and implementation of scalable data pipelines usingAzure Data Services, Databricks, and Python.
- Architect and optimizeETL workflowsto ensure data integrity and efficiency.
- Collaborate withdata scientists, analysts, and business stakeholdersto understand data requirements and provide robust solutions.
- Manage and mentor a team of data engineers, providing guidance on best practices and performance optimization.
- Implement and enforcedata governance, security, and compliancestandards.
- Troubleshoot and resolve data pipeline issues, ensuring high availability and reliability.
- Optimizedata storage, transformation, and retrievalfor large datasets in a cloud environment.
Required Skills & Experience:
- 8+ years of experience inData Engineering, with at least2+ years in a lead role.
- Strong hands-on experience withAzure Data Services(Azure Data Factory, Azure Synapse, Azure SQL, Azure Data Lake).
- Expertise inDatabricksandSparkfor big data processing.
- Proficiency inPythonfor data manipulation and pipeline development.
- Deep understanding ofETL/ELT processesand best practices.
- Experience withCI/CD pipelinesandDevOps practicesfor data engineering.
- Strong knowledge ofdata modeling, warehousing concepts, and performance tuning.
- Excellent problem-solving, analytical, and communication skills.
