About the Role
We are seeking experienced software engineers to help evaluate how well AI models handle real software engineering problems. This is a short-term, high-impact role where your engineering expertise will directly influence the training and assessment of next-generation AI systems built for developers.
You'll be working with a global AI company on a specialized project focused on analyzing code produced by Large Language Models (LLMs). The goal: make these models better at reading, writing, and understanding real-world code.
What You'll Do
- Assess AI-generated code for quality, correctness, readability, and efficiency
- Compare multiple code outputs and rank them using clear guidelines
- Review code diffs from actual GitHub projects and judge their effectiveness
- Write concise explanations to support each ranking decision
- Identify edge cases or confusing outputs that indicate AI weaknesses
- Work with a team of experts to improve evaluation standards and datasets
Must-Have Requirements
Please do not apply unless you meet all of the following baseline requirements:
- Experience:
- 5+ years of overall professional software engineering experience (experience working as a data scientist will not be considered)
- 2+ years working full-time as a Fullstack Engineer at a top-tier tech product company
- Companies include Google, Datadog, Shopify, Meta, Canva, Amazon, and others.
Note: Contract-only or part-time roles will not be considered as experience
- You are skilled at reading and analyzing Git-style diffs
- You can write clear, structured reasoning to explain technical choices
- You follow rubrics and guidelines to ensure fair, structured evaluations
Nice to Have
- Exposure to LLM-generated code or prior experience evaluating model outputs
- Degree from a top university (not required, but preferred)
- Background in developer tools, automation, or open-source contributions
- Experience with AI research or evaluation workflows
Engagement Details
- Type: Contract (independent contractor)
- Duration: 1 month to start (with possible extensions)
- Hours: Flexible work hours with commitment of 10-20 hours/week (must have some overlap with Pacific Time)
- Compensation: $50–$150/hour (based on experience and skill level)
- Start Date: Immediate openings available (next week)
About Turing:
Turing is one of the world's fastest-growing AI companies, pushing the boundaries of AI-assisted software development. Our mission is to empower the next generation of AI systems to reason about and work with real-world software repositories. You'll be working at the intersection of software engineering, open-source ecosystems, and frontier AI.
Why Join Turing
- Be part of a cutting-edge AI project helping shape how developers work with code
- Collaborate with world-class AI researchers and engineers
- Apply your skills in a meaningful way—on real software, not theoretical examples
