Summary:
As an LLM Evaluation Expert specializing in Coding, you will play a crucial role in assessing and improving our language models coding capabilities. Your expertise will be instrumental in evaluating LLM-generated code responses, making high-level judgments, and setting the standard for what constitutes excellent AI-assisted coding.
Responsibilities:
- Critically analyze and evaluate code responses generated by our LLMs across various programming languages and paradigms.
- Exercise expert judgment to select the most appropriate and efficient code solutions from multiple LLM-generated options.
- Make informed decisions on behalf of our customers, ensuring that selected code meets industry standards, best practices, and specific client needs.
- Develop and write coding demonstrations to illustrate "what good looks like' in AI-generated code, setting benchmarks for quality and efficiency.
- Provide detailed feedback and explanations for your evaluations, helping to refine and improve the LLM's understanding and output.
- Collaborate with the AI research team to identify areas for improvement in the LLM's coding capabilities.
- Stay abreast of the latest developments in software engineering, coding standards, and AI to ensure our evaluations remain cutting-edge.
Requirements:
- Advanced degree in Computer Science, Software Engineering, or a related field.
- Extensive experience in software development across multiple programming languages and paradigms.
Required Skills:
- Demonstrated ability to critically evaluate code quality, efficiency, and adherence to best practices.
- Strong analytical and decision-making skills, with the ability to make complex judgments under ambiguous circumstances.
- Excellent written and verbal communication skills, with the ability to explain technical concepts clearly.
- Experience in technical writing, particularly in creating coding examples or tutorials.
Preferred Skills:
- Previous experience working with or evaluating AI systems, particularly in the context of code generation.
- Familiarity with a wide range of software development methodologies and architectural patterns.
- Understanding of machine learning concepts, particularly as they apply to natural language processing and code generation.
- Experience in creating or contributing to coding standards or style guides.
#AditiConsulting
#26-01387
