Apache Spark

Summary: In the role of Responsible Artificial Intelligence Technology Lead, a typical day involves overseeing the development and implementation of artificial intelligence solutions that prioritize ethical considerations and responsible use. This position requires guiding teams to design and deploy AI technologies that comply with established ethical standards and regulatory frameworks. The role includes fostering collaboration across various departments to ensure alignment with organizational values and advocating for the adoption of responsible AI practices throughout the project lifecycle. The individual will also engage in strategic planning and decision-making to promote transparency and accountability in AI initiatives. Roles & Responsibilities: - Expected to be an SME, collaborate and manage the team to perform.- Responsible for team decisions.- Engage with multiple teams and contribute on key decisions.- Provide solutions to problems for their immediate team and across multiple teams.- Lead the development and implementation of ethical AI frameworks within projects to ensure compliance with regulatory and organizational standards.- Facilitate cross-functional communication to align AI initiatives with responsible use principles and promote shared understanding among stakeholders.- Mentor junior team members by providing guidance on ethical AI practices and supporting their professional growth. Professional & Technical Skills: - Must To Have Skills: Proficiency in Apache Spark.- Strong knowledge of distributed computing concepts and experience in processing large-scale data using Apache Spark.- Experience in designing and optimizing data pipelines to support AI and machine learning workloads.- Familiarity with data governance and security practices relevant to AI systems.- Ability to troubleshoot and resolve performance issues in Spark applications.- Experience with integrating Apache Spark with other big data technologies and cloud platforms