Establish metrics to benchmark AI model issues and quantify improvements. Develop datasets, analyze and run evaluations for engineers to evaluate and improve Earnin s product suite.
Use technical judgment to drive project delivery, challenge proposals and identify risks
Collaborate closely with Data Scientists and Machine Learning Engineers
Improve tools through data analysis, technical expertise, and presentations Enhance operational workflows via process improvements and identification of automation opportunities
Evaluate and optimize high-quality prompts for GenAI applications
Perform systematic evaluation of prompts and LLMs across different use cases
Recommend ways to measure performance and success of prompts
Communicate capabilities, strengths and weaknesses of GenAI capabilities to stakeholders in a clear and concise manner for better decision making
Stay up to date with the latest developments in GenAI (especially LLMs), in both academia and the industry, including closed-source and open-source models
WHAT WERE LOOKING FOR:
Strong mathematical, statistical or data science background
Strong understanding of ML lifecycle, LLMs, prompt engineering techniques, GenAI technologies and frameworks
Keen interest in the development of GenAI models and applications
Hands on experience in machine learning, predictive analytics and statistical modeling would be an advantage
Adaptable and sensitive to the evolving nature of GenAI, understanding implications and able to think ahead
Able to take on projects tasks independently from end-to-end
Proficient in Python, with strong coding practices in GenAI domain including prompt libraries and prompt template reusability
Demonstrably excellent verbal and written communication skill
