Role: Japanese AI Quality Evaluator / AnnotatorExperience: 1+ Year
Engagement Type: Short-Term Contract (2 Months)
Work Mode: Remote
Availability: Full-Time (8 Hours/Day) with 4 hours overlap with PST time zone
Key Responsibilities- Evaluate AI-generated responses in Japanese for accuracy, personalization, and quality.
- Perform Side-by-Side (SxS) model comparisons to identify differences in response quality.
- Design creative prompts and multi-turn conversations to test AI model capabilities.
- Analyze nuanced AI responses and evaluate personalization relevance.
- Provide structured feedback and detailed annotations for model improvements.
- Write clear rationales explaining why one response is better than another.
Required Skills & Qualifications- Japanese Language Proficiency: Ability to read and write Japanese fluently with high comprehension.
- Experience in AI evaluation, data annotation, content moderation, or similar roles is highly preferred.
- Bachelor's degree in fields such as Policy, Law, Ethics, Linguistics, Journalism, Computer Science, or other analytical disciplines (or equivalent experience).
- Strong analytical and critical thinking skills.
- Ability to identify incorrect personalization, forced connections, and poor inferences in AI responses.
- High attention to detail when reviewing AI responses.
- Strong written communication skills to provide clear explanations and feedback.
- Ability to work independently in a remote environment.
Special Requirements- Personal Google Account Usage: Candidates must be willing to use their primary personal Google account and enable personal data sources for authentic evaluation.
- Schedule Flexibility: Ability to work in a global 24/7 team environment.
- Technical Setup: Desktop/Laptop with stable high-speed internet.
Preferred Experience- AI Quality Evaluation
- Data Annotation / Labeling
- Prompt Engineering
- Content Moderation
