Video Eval Annotator

Role Overview

A US based AI company that is at the forefront of AI research is looking for a detail-oriented and analytically consistent Annotator to evaluate visual and creative content using structured rating frameworks and detailed guidelines.

In this role, you will assess AI-generated and real-world media (images and videos) for quality, realism, composition, and adherence to defined evaluation standards. Your work will directly contribute to improving cutting-edge AI systems.

This position requires strong visual judgment, structured reasoning, and consistency across high-volume evaluation tasks.

Job Responsibilities

• Evaluate visual and media content using structured guidelines and predefined rating frameworks.
• Provide clear, well-structured written justifications to support evaluation decisions.
• Ensure consistency, accuracy, and attention to detail across high-volume annotation tasks.
• Identify quality gaps related to realism, composition, motion, lighting, and overall production standards.

Job Requirements

• 3+ years of professional work experience
• Media/agency experience is mandatory
• Hands-on experience working directly with designers, photographers, or videographers (mandatory)
• Experience with AI image or video generation tools is a strong plus
• Strong understanding of motion realism, lighting, camera movement, composition, and production quality
• Strong analytical, logical reasoning, and research skills and excellent communication and collaboration abilities, with proficiency in English.
• Incorporate feedback from LLM performance metrics into training data creation.
• Ability to commit to 40 hours per week and candidates who can join immediately preferred.

Required Qualification:

Bachelor's degree or higher in:

• Design, Visual Communication, or Fine Arts
• Film, Media, Animation, or VFX
• Photography or Cinematography
• Architecture or Industrial Design
• Creative Writing or equivalent professional experience in content creation, media production, or content review.

Application Process

• Shortlisted candidates will complete an assessment.
• Once the assessment is cleared, candidates can start.

Obs:

Please note that we'll be using a new customized vetting process for this project.

• The task involves reviewing AI-generated videos and identifying hallucinations/anomalies such as object morphing, disappearing objects, physically impossible movements, etc.

• The primary skill requirement is strong attention to detail — deep technical or video editing expertise is not mandatory.

• Video durations may vary from 5 seconds to 60 seconds depending on the workflow complexity.

• The team is working on a new customized Vetsmith flow that will include sample video evaluation tasks along with golden set answers for more objective screening and quality control.

• Initial Vetsmith setup may take around 2–3 days, but this is expected to significantly improve quality and reduce manual review efforts downstream.

• It is expected to scale up to 600+ annotators overall, with aggressive ramp-up timelines targeting early June.

• For sourcing, please prioritize candidates with annotation, moderation, QA, content review, or other detail-oriented experience

Location:

India

Brazil

Austria

• Bulgaria

• Croatia

• Cyprus

• Denmark

• Estonia

• Finland

• Germany

• Gibraltar

• Iceland

• Latvia

• Liechtenstein

• Lithuania

• Malta

• Norway

• Poland

• Romania

• Slovakia

• Slovenia

• Switzerland