Evaluation Science Brief
What Does an Agent Evaluation Actually Support?
A benchmark score is often taken as evidence that an agent is ready for a job. For example:
System A achieved 74% on benchmark B under configuration C.
Using this result for workflow X takes four further steps, which argument-based validity calls the scoring, generalization, extrapolation, and decision inferences [1,2]. A result is only as strong as the weakest of them.
The four steps
| Step | Question | Evidence |
|---|---|---|
| Scoring | Did the grader judge each run correctly? | Grader validation [3,10] |
| Generalization | Do the tasks and runs represent the domain? | A defined task frame and an uncertainty estimate [4–6] |
| Extrapolation | Does the benchmark resemble the target setting? | A direct comparison of the two settings [7,8] |
| Decision | Is the expected performance good enough for the action? | A threshold based on the cost of failure and the alternatives [9] |
Step 1: Record what was evaluated
For agents, the model name alone does not identify the system. Record the model version, scaffold, prompts, tools, permissions, and date as the system. Record the task-set version, environment, grader, run budget, and retry policy as the evaluation. Keeping the two apart shows whether a change in the score came from the agent or from how it was evaluated.
Step 2: Check the four steps in order
Start with scoring and move down the table. For scoring, test the grader on outputs whose status is known, as described in our grader brief. Agents can also exploit the evaluation environment and reach a rewarded state without doing the task [10], so keep trajectories when this is plausible. For generalization, define which tasks could enter the evaluation, state how timeouts, crashes, and partial completion are counted, and report the uncertainty. Report how failures are distributed as well, because two agents with the same mean can fail in very different ways [6]. For extrapolation, compare the benchmark with the target workflow, as in our workflow brief. For the decision, set the threshold from the consequences of error, as in our decision brief.
Step 3: Stop at the first step without evidence
The conclusion should go only as far as the evidence does. For example:
Decision being considered:
Use A for workflow X
Scoring ✓
Generalization ✓
Extrapolation to workflow X ?
Decision ✕
SUPPORTED CONCLUSION:
A's performance has been established in the defined
evaluation domain under the evaluated configuration.
NEXT STUDY:
Compare the benchmark with workflow X and specify
the decision threshold.Evaluation Evidence Record
| Field | Record |
|---|---|
| Decision | What is the evaluation being used to answer? |
| System | Model, scaffold, prompts, tools, permissions, versions, date |
| Evaluation | Task-set version, environment, grader, run budget, retry policy |
| Task frame | Which tasks could enter the evaluation, and how they were selected |
| Run policy | How timeouts, crashes, retries, and partial completion are counted |
| Evidence by step | Scoring, generalization, extrapolation, decision |
| Supported conclusion | Strongest conclusion the evidence supports |
| Next study | What would address the first unsupported step |
References
- Kane MT. Validating the Interpretations and Uses of Test Scores. Journal of Educational Measurement. 2013;50(1):1–73.
- Cook DA, Brydges R, Ginsburg S, Hatala R. A contemporary approach to validity arguments: a practical guide to Kane's framework. Medical Education. 2015;49(6):560–575.
- Zhu Y, Jin T, Pruksachatkun Y, et al. Establishing Best Practices for Building Rigorous Agentic Benchmarks. Advances in Neural Information Processing Systems. 2025.
- Yao S, Shinn N, Razavi P, Narasimhan K. τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains. International Conference on Learning Representations. 2025.
- Mizrahi M, Kaplan G, Malkin D, et al. State of What Art? A Call for Multi-Prompt LLM Evaluation. Transactions of the Association for Computational Linguistics. 2024;12:933–949.
- Rabanser S, Kapoor S, Kirgis P, et al. Towards a Science of AI Agent Reliability. Proceedings of the 43rd International Conference on Machine Learning. 2026.
- Raji ID, Denton E, Bender EM, Hanna A, Paullada A. AI and the Everything in the Whole Wide World Benchmark. Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks. 2021.
- D'Amour A, Heller K, Moldovan D, et al. Underspecification Presents Challenges for Credibility in Modern Machine Learning. Journal of Machine Learning Research. 2022;23(226):1–61.
- Kapoor S, Stroebl B, Siegel ZS, Nadgir N, Narayanan A. AI Agents That Matter. Transactions on Machine Learning Research. 2025.
- Shao J, Chen H, Zhang W, Pan M, Luo B. Do Agent Benchmarks Measure Capability? Protocol Validity in the Age of Agentic AI. arXiv:2607.22368. 2026.