BrainBrowser builds the safety and alignment layer for frontier AI systems working with leading AI labs and foundational model developers on supervised fine-tuning data, RLHF, adversarial red teaming, and safety evaluations. Our work answers the question capability can't: what happens when the model fails, and how do we prevent it This role contributes directly to our SFT data pipeline.
The Role
Review and evaluate AI-generated code samples against a structured 6-point rubric. Assess prompt realism, vulnerability plausibility, security protocol accuracy, security correctness, functional correctness, and completeness. Flag issues with written justification and provide corrected code.
What Youll Do
- Review coding tasks (context + completion pairs)
- Evaluate each sample across defined rubrics
- Flag issues with written justification; write corrected code where needed
- Work from a detailed rubric acceptance criteria are unambiguous
Who We Need
- 3+ years software engineering
- Strong in at least 3 of: Python, JavaScript/TypeScript, Java, C#, C, C++, Rust, SQL, Bash
- Solid understanding of security engineering injection, auth flaws, deserialization, path traversal, SSRF, race conditions, and others
- Can distinguish a security bug from a functional bug these are evaluated separately
- Clear, precise written English
Engagement: Fixed price per approved violation identified | 10 samples to start calibration batch
Apply with: Tell us the languages you're strongest in and one CWE or security protocol you know deeply with a real example.
This job is provided by Shine.com
