Data Engineer

Position: Azure Databricks Jr/Sr Developer

Experience: 7-12 Years

Location: Gurgaon

Job Summary

We are looking for a highly skilled Azure Databricks Developer with 7-12 years of experience in Data Engineering, Big Data technologies, and cloud-based data platforms. The ideal candidate should have hands-on expertise in Azure Databricks, PySpark, Azure Data Factory (ADF), Delta Lake, and building scalable data pipelines in Azure Cloud environments.

Key Responsibilities

• Design, develop, and maintain scalable data ingestion and transformation pipelines using Azure Databricks and PySpark.
• Build and optimize ETL/ELT workflows for large-scale structured and unstructured datasets.
• Develop data solutions leveraging Azure Data Factory, Azure Data Lake Storage (ADLS), and Azure Synapse Analytics.
• Implement Delta Lake architecture for reliable and high-performance data processing.
• Work with business stakeholders, data architects, and analysts to understand data requirements.
• Perform data modeling, cleansing, validation, and transformation activities.
• Optimize Spark jobs for performance, scalability, and cost efficiency.
• Implement CI/CD pipelines and deployment automation for Databricks solutions.
• Monitor and troubleshoot production data pipelines.
• Ensure data governance, security, and compliance standards are followed.
• Participate in architecture discussions and provide technical recommendations.