Job Title: Data Analyst
Location: Remote / Global
About the Role
We are looking for a data-driven professional to join our analytics and quality team. In this role, you will analyze the performance of AI agents across finance-sector tasks to uncover root causes of failure and drive improvements in design, evaluation, and benchmarking frameworks.
Key Responsibilities
- Conduct statistical failure analysis to identify recurring patterns in AI agent performance across prompts, rubrics, templates, and file types
- Investigate root causes behind performance issues in task execution, framework design, or data complexity
- Perform multi-dimensional analysis across finance sub-domains, task categories, and input types
- Develop dashboards and reports highlighting performance trends, failure clusters, and improvement opportunities
- Recommend enhancements to task design, quality rubrics, and evaluation metrics based on data findings
Required Qualifications
- Strong understanding of statistical analysis, hypothesis testing, and pattern recognition
- Proficiency in Python (pandas, scipy, matplotlib/seaborn) or R for data analytics
- Hands-on experience in exploratory data analysis (EDA) and turning data into actionable insights
- Familiarity with AI/ML evaluation techniques and quality assessment metrics
- Proficiency in Excel, SQL, and visualization tools such as Tableau or Looker
Preferred Qualifications
- Prior experience in AI/ML model evaluation, data labeling quality, or automation analysis
- Knowledge of finance-sector concepts or willingness to learn finance domain structures
- Experience with multidimensional failure analysis or benchmarking studies
- Understanding of large-scale evaluation frameworks or test dataset design
Equal Opportunity Statement
We welcome applicants from all backgrounds. Reasonable accommodations are available for qualified individuals with disabilities.
