Type, Location,
Full Time @ Anywhere in India
Desired Experience
4+ years
About the Role
We are looking for a semi-technical Task Runner / QA Specialist to execute, validate, and document AI agent tasks on our platform. This role acts as a critical bridge between domain experts designing AI tasks and the engineering team building automated infrastructure.
This is an MVP-stage role where tooling is still evolving. The ideal candidate is comfortable working with incomplete systems, filling operational gaps with technical competence, and contributing directly to shaping future tooling and workflows.
Key Responsibilities:-
Task Execution & Testing
- Pick up assigned tasks from the queue, mark them in-progress, and execute them following detailed protocols
- Seed and configure test environments with required datasets
- Run AI agent test sessions and assess whether tasks are completable, well-scoped, and appropriately challenging
- Validate task logic and surface any execution gaps or inconsistencies
Quality Assurance & Validation
- Write concise, structured test reports summarizing AI performance, issues identified, and improvement recommendations
- Package test outputs including recordings, screenshots, logs, and documentation for review by domain experts
- Collaborate asynchronously with domain experts to iterate on task quality
- Identify and flag UI bugs, environment issues, and ambiguities in task specifications
Technical Handling & Documentation
- Convert plaintext or unstructured data schemas into structured JSON aligned with predefined schemas
- Organize and maintain JSON seed data, task configurations, and test outputs for reuse and automation
- Develop automated verifiers for validated tasks and integrate them into the computer-use infrastructure
- Document completed tasks thoroughly for inclusion in formal evaluation and production pipelines
Required Skills & Qualifications:-
- Strong comfort with JSON (reading, writing, validating, and schema-based structuring)
- Experience using command-line tools, Git, and basic project setup workflows
- Ability to follow complex, multi-step technical instructions with high precision
- Strong attention to detail and a QA-oriented mindset for identifying edge cases
- Experience capturing screen recordings, screenshots, and structured documentation
- Self-directed, reliable, and comfortable working asynchronously in a fast-paced, early-stage environment
- Availability for 1020 hours per week, with consistent availability during assigned time blocks
