Beyond Building the Most Difficult Benchmarks

As frontier models quickly improve their scores on existing benchmarks, more and more new benchmarks are maintaining discriminative power by actively looking for harder cases. BrowseComp, for example, is built around making answers hard to find but easy to verify, and then filtering out questions that frontier models can already solve.[1] More recently, Terminal-Bench-Science made difficulty targeting even more explicit: its task proposal rubric directly asks for tasks where frontier models have only a 10–20% success rate at release, and every new release recalibrates difficulty based on the frontier models at the time; in version 0.1 released in 2026, the strongest system (Claude Opus 5) had a 30% resolution rate across 70 scientific workflow tasks.[2] Some of these benchmarks contain tasks that take human experts a long time to complete, while others allow agents to use reasoning horizons, large numbers of tool calls, and very high inference budgets that are basically unrealistic in actual use. From the perspective of frontier capability measurement, this design makes sense: harder tasks let us keep observing the capability boundaries of scaling, reasoning, search, long-horizon autonomy, etc.

But this benchmark construction strategy has a very specific (and narrow) evaluation target. It mainly answers one question: under a particular compute budget and execution setup, what is the upper bound of the problems an agent can solve? It cannot tell us whether these systems can reliably complete everyday tasks for ordinary users or researchers, i.e., whether in ordinary work they miss critical information, forget earlier constraints, call the wrong tool, continue generating something that looks plausible after a failure, or give obviously different answers across runs on the same task, etc.

Unfortunately, these two questions are now basically mixed together. People use scores on difficult benchmarks to infer model capability, while “capability” is also used to include the model’s ability to complete everyday tasks. Of course, the gap between benchmark scores and actual user experience has many sources (deployment scaffolding, context management, prompts, user expectations are all part of it). Evaluations closer to real work have already given us some fairly direct evidence. In AssistantBench, agents performed poorly on realistic tasks requiring multi-step web browsing, with low precision and hallucinated facts.[3] OSWorld also observed a large performance gap between agents and humans on real computer-use workflows.[4] More recently, DRBench brought the same problem into enterprise deep research and found that agents still had clear problems with factual accuracy and relevant-information recall.[5] These results show a fairly simple point: an agent being able to show very strong capability on some difficult tasks does not mean that it can already complete ordinary work reliably.

τ-bench, on the other hand, exposes some measurement problems. The strongest function-calling agents at the time already had success rates below 50% on customer-service tasks, but when the same tasks and the same agents were evaluated using pass^k (i.e., the probability that all k independent runs succeed), the scores dropped much further.[6] Same benchmark, same agent, just a different estimand, and the conclusion becomes completely different.

This also tells us something about failures in everyday use. They often do not look like complete task failure, but are more subtle: a system may get some information right but miss the decisive evidence; it may eventually complete the task but use far more steps and time than a human; or its success rate may look high on a single run but quickly reveal instability under repeated execution. These properties cannot be measured by simply increasing task difficulty, because the metric being reported is not asking these questions in the first place.

It is also inevitable that a large amount of research effort goes toward harder cases. On one hand, this is a very clear research target: as long as the strongest current models cannot solve the benchmark, the new benchmark has headroom and can continue to distinguish frontier systems. On the other hand, the contribution of this kind of work is also very easy to express legibly. You do not need to first explain what work a model actually needs to do within a particular kind of knowledge work, how the task distribution should be sampled, or what kind of agent behavior should count as actually useful. “The strongest current models cannot solve this” is already a claim with its own stopping criterion, and because of that, it is also easier to review, reproduce, and compare.

The situation is almost exactly the opposite if we want to study everyday use. There is no simple and straightforward story here (largely because many basic things have still not been clearly defined, and there is no real consensus). For example, what actually constitutes a meaningful everyday task distribution? Should we sample from real user requests, workflows, or existing task repositories? How should we compare the severity of different failures? If an agent is correct on 95% of tasks, but the remaining 5% of errors are almost impossible for the user to detect, how should we compare it with a system that is slightly less accurate but clearly exposes its uncertainty? For a workflow that requires dozens of consecutive steps, should we report average success per step, or the probability that the entire trajectory succeeds? When a system gains a few percentage points by using ten times more compute, how should that enter the evaluation?

The good news is that none of these questions is completely new. Their machinery already exists, just mostly outside agent evaluation. The relationship between per-step and trajectory-level success is the standard series-system problem in reliability engineering; instability under repeated execution is a variance decomposition problem in generalizability theory (i.e., how much variance comes from the task, the run, the prompt, etc.); “what exactly is the quantity represented by the number we report?” is the kind of problem the estimand framework in clinical trials was built to deal with; and the difference between silent errors and flagged errors corresponds to very different error cost structures in diagnostic testing. These tools already exist. They just have not been systematically ported into agent evaluation.

What is actually missing is not the methods, but treating “what questions should we be asking?” as a research object itself. Only when we know what questions we should ask (and how we should answer them) can we generate enough evidence to support actually handing work over to agents.

Of course, we still need frontier benchmarks, but agent evaluation should also start moving toward a more complete scientific approach. Beyond continuing to search for harder tasks, we urgently need to build the basic evidence generation science for everyday agent use, and form clearer consensus and a framework that people can follow around this set of questions that are still poorly defined.

References

1. Wei J, Sun Z, Papay S, McKinney S, Han J, Fulford I, et al. BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents. arXiv:2504.12516, 2025.

1. Terminal-Bench-Science Team. Terminal-Bench-Science: Evaluating AI Agents on Research Workflows Across Scientific Domains. 2026. doi:10.5281/zenodo.22110253.

1. Yoran O, Amouyal SJ, Malaviya C, Bogin B, Press O, Berant J. AssistantBench: Can Web Agents Solve Realistic and Time-Consuming Tasks? In: Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024:8938–8968. doi:10.18653/v1/2024.emnlp-main.505.

1. Xie T, Zhang D, Chen J, Li X, Zhao S, Cao R, et al. OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments. Advances in Neural Information Processing Systems. 2024.

1. Abaskohi A, Chen T, Muñoz-Mármol M, Fox C, Ramesh AV, Marcotte É, et al. DRBench: A Realistic Benchmark for Enterprise Deep Research. arXiv:2510.00172, 2025.

1. Yao S, Shinn N, Razavi P, Narasimhan K. τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains. International Conference on Learning Representations. 2025.