Job DescriptionAbout the RoleWe are seeking highly skilled Data Engineers with 4+ years of hands-on experience in Databricks and strong expertise in data engineering, ML pipelines, and cloud-based data platforms. The ideal candidate will design, build, and maintain large-scale data processing systems, enabling high-quality data flow and analytics for business insights.
Key Responsibilities- Design, develop, and optimize ML/data transformation pipelines using Python,Pandas, PySpark, and Apache Spark.
- Build and manage scalable data architectures for handling large and complex datasets.
- Work extensively on Azure Databricks and Delta Lake, ensuring efficient data storage and processing.
- Collaborate with cross-functional teams to deliver data solutions across multiple domains.
- Implement best practices in data modeling, data warehousing, and data governance for both on-prem and cloud environments.
- Continuously evaluate emerging technologies and tools to enhance data engineering capabilities.
- Troubleshoot, optimize, and ensure performance of data pipelines and analytical systems.
- Take ownership of technical deliverables, contributing to project leadership and independent execution.
Required Skills & ExperienceMandatory:
- Strong hands-on experience with Databricks and Delta Lake.
- Proficiency in Python and Pandas(compulsory).
- Strong experience in PySpark and Apache Spark.
- Experience in ML pipelines, data integration, and data pipeline optimization.
- Solid understanding of data modeling, data warehousing, and distributed systems.
- Ability to manage and transform large-scale, multi-dimensional datasets.
- Strong problem-solving, analytical, and process optimization skills.
- Up-to-date with modern data engineering tools and cloud data ecosystems.
