About The Opportunity
A fast-scaling AI research and evaluation lab in the Generative AI & Spatial Reasoning domain, we build and benchmark next-gen video-based reinforcement learning (RL) agents that perceive, plan, and act in 3D environments. Our work powers real-world embodied AI applications—from robotics navigation to autonomous systems—by rigorously testing how models understand spatial relationships, motion logic, and scene dynamics from video input. We're hiring a Spatial Reasoning Evaluation Specialist to design, execute, and scale high-fidelity human-in-the-loop evaluations for video RL models.
Role & Responsibilities
- Design and implement standardized evaluation protocols to assess spatial reasoning performance of video RL agents across tasks like path planning, object interaction, and scene navigation.
- Annotate, curate, and label video datasets to create ground-truth benchmarks for agent evaluation, with emphasis on temporal-spatial consistency and action causality.
- Run human-in-the-loop experiments comparing agent output against human baseline performance using structured scoring rubrics and UI tools.
- Identify failure modes, edge cases, and spatial reasoning gaps in agent behavior and document findings for research and engineering teams.
- Collaborate with ML researchers to iterate on evaluation metrics, refine task design, and improve benchmark robustness.
- Build and maintain evaluation dashboards to track agent performance trends, score distributions, and spatial reasoning drift over time.
Skills & Qualifications
- Must-Have
- PyTorch
- OpenCV
- Video annotation tools (e.g., CVAT, Labelbox)
- Spatial reasoning task design
- Human-in-the-loop evaluation frameworks
- Reinforcement learning basics (e.g., PPO, DQN, env interaction)
- Python scripting for data validation & reporting
- 3D scene understanding (e.g., coordinate frames, depth maps, camera poses)
- Preferred
- Experience with embodied AI platforms (e.g., Habitat, AI2 Thor, Unity ML-Agents)
- Familiarity with RLlib or RL frameworks for agent benchmarking
- Background in cognitive science or psychophysics of spatial reasoning
Benefits & Culture Highlights
- Work directly with cutting-edge video RL models shaping the future of embodied AI.
- On-site innovation lab environment in India with access to high-end annotation tools and compute infrastructure.
- Opportunity to co-author research evaluations and contribute to public benchmarks in spatial reasoning.
Skills: design,annotation,agents,research,video,learning,models,edge,3d,evaluations
