Follow us

Research & Commons

Tools

Evaluation tools EvalScience builds together with experts from the field.
The four-field record: represented activity, tested setting, required work product, and evaluated result, with what each field asks, what to record, where to look, an APEX-SWE example, and the rule for comparing the required work product with the evaluated result.
Figure 1 from the paper · The four-field record
Framework · Benchmark design

Designing Benchmarks for Knowledge Work

A framework for designing and reporting benchmarks of knowledge work. Besides its tasks, environment, and metric, a benchmark should state four fields: the work it represents, the setting it tests, the work product the task requires, and the part of that product it scores. The paper derives each field from research on how knowledge work is organized and applies the framework to GDPval, OfficeQA Pro, and APEX-SWE.

In development · Health care

HEED: Health AI Evaluation and Design

HEED is a new evaluation tool for health care AI agents, built from the ground up by an international group of experts. It follows the approach we used for the Health Care AI Chatbot Evaluation Framework (HAICEF), our published systematic review of evaluation frameworks for health care chatbots: clinicians, researchers, developers, policymakers, patients, and patient advocates review each question in the tool and decide whether to keep, revise, or remove it.

More than 40 experts have joined the HEED Expert Group. Members are credited on the HEED paper through the Expert Group author.

Join as an expert

Tell us about your background, what you work on, what you could contribute, and what you are interested in. Include publications, who has cited your work, and any other evidence of your expertise. We read every response and reply by email.

Join as an expert

Suggest a tool or open standard the field needs

Tell us which evaluation tool or open standard is missing, what it should do, and who needs it. Suggestions shape what we build next with the community.

Suggest a tool