Follow us

Research & Commons

Notes & Briefs

Evaluation Science Briefs: concise, source-linked articles on how to design, interpret, and use evidence from agentic system evaluations.

Featured / Start here

How to check whether a benchmark score holds up in the workflow where an agent will actually be used.

4 min read

  1. 01

    What Does an Agent Evaluation Actually Support?

    How to check, step by step, how far a benchmark score can be taken toward a deployment decision.

    5 min read

  2. 02

    How Do You Know Your Agent Grader Works?

    How to test a grader on outputs with known answers before using it to compare agents.

    5 min read

  3. 03

    Is This Agent Result Reproducible?

    How to test whether a result holds up beyond reruns of the same setup.

    4 min read

  4. 04

    Is This Agent Good Enough for the Decision?

    How to set a threshold and a decision rule before comparing a score against them.

    4 min read

  5. 05

    What Should an Agent Evaluation Report So Someone Else Can Use It?

    What to put next to a score so that someone else can check and reuse it.

    3 min read

Event notes

  1. Boston

    Who Checks AI's Work? Notes from Boston

    Generating tasks, environments and papers with AI is now easy; checking them is the bottleneck. From the October 28 Boston working group.

  2. Bay Area

    Bay Area Research Gathering: Summary

    What researchers from AI labs, technology companies, and startups said agent evaluation needs in order to become a scientific practice.

  3. Boston

    Boston Working Group, September 2026

    What researchers from MIT, Harvard, Northeastern, Brown, UMass Amherst, and industry see as the main bottlenecks in agent evaluation.