Skill: Pyspark + Databricks
Experience: 6 to 9 Years
Location : AIA - Gurgaon
Job Summary
We are seeking a highly skilled Technical Lead with 12 to 16 years of experience to join our dynamic team. The ideal candidate will have expertise in Databricks Workflows Unity Catalog and Databricks with a preference for experience in Research and Development. This hybrid role offers the opportunity to work on cutting-edge projects during day shifts with no travel required.
Responsibilities
- Lead the design and implementation of scalable data solutions using Databricks Workflows to enhance data processing capabilities.
- Oversee the integration of Databricks Unity Catalog to ensure seamless data governance and security across platforms.
- Provide technical expertise in Databricks to optimize performance and drive innovation in data analytics.
- Collaborate with cross-functional teams to deliver high-quality solutions that meet business requirements and objectives.
- Develop and maintain comprehensive documentation for data workflows and processes to facilitate knowledge sharing.
- Mentor and guide junior team members to foster a culture of continuous learning and improvement.
- Conduct regular code reviews to ensure adherence to best practices and coding standards.
- Analyze complex data sets to identify trends and insights that support strategic decision-making.
- Implement robust testing frameworks to ensure the reliability and accuracy of data solutions.
- Coordinate with stakeholders to gather requirements and translate them into technical specifications.
- Drive the adoption of new technologies and methodologies to enhance the teams capabilities and efficiency.
- Ensure compliance with industry standards and regulations to protect data integrity and privacy.
- Monitor system performance and troubleshoot issues to maintain optimal operation of data platforms.
Qualifications
- Possess extensive experience in Databricks Workflows and Unity Catalog demonstrating a deep understanding of their functionalities.
- Have a strong background in data engineering and analytics with a proven track record of delivering successful projects.
- Exhibit excellent problem-solving skills and the ability to work effectively in a hybrid work model.
- Demonstrate proficiency in programming languages such as Python or Scala relevant to Databricks environments.
- Show a keen interest in Research and Development with the ability to apply domain knowledge to data solutions.
