Job Title-Data Engineer
Location: PAN India
Azure Data Engineer – Databricks, ADF, ADLS Gen2 Job Description
We are looking for a skilled
Azure Data Engineer with strong hands-on experience in
Azure Databricks,
PySpark,
Azure Data Factory (ADF), and
Azure Data Lake Storage Gen2 (ADLS Gen2). The ideal candidate will have experience designing, developing, and maintaining scalable, production-grade data pipelines and working with large-scale data processing solutions on Microsoft Azure.
Key Responsibilities
- Design, develop, and maintain scalable ETL/ELT pipelines using Azure Data Factory (ADF).
- Build and optimize data processing solutions using Azure Databricks and PySpark.
- Develop and manage data lakes using ADLS Gen2.
- Implement data transformation, cleansing, and integration processes.
- Optimize Spark jobs for performance, scalability, and cost efficiency.
- Work with Delta Lake for reliable and high-performance data processing.
- Collaborate with business analysts, data scientists, and stakeholders to deliver data solutions.
- Monitor, troubleshoot, and support production data pipelines.
- Ensure data quality, governance, and security best practices.
- Participate in code reviews and implement CI/CD best practices for data engineering solutions.
Mandatory Skills
- Strong hands-on experience with Azure Databricks.
- Excellent experience in PySpark with real-time project exposure.
- Experience with Azure Data Factory (ADF).
- Experience with Azure Data Lake Storage Gen2 (ADLS Gen2).
- Strong Azure Data Engineering experience.
- Experience building scalable, production-grade data pipelines.
- Strong SQL skills.
- Knowledge of ETL/ELT concepts and data warehousing
Skills: adls,gen2,adf,data bricks
