In October, EvalScience hosted an invitation-only gathering in the Bay Area with PhD students, postdocs, and researchers from AI labs, technology companies, and startups. The discussion focused on what agent evaluation needs in order to become a scientific practice.
An agenda developed outside any single company
Participants from industry labs described a gap between the pace of model development and the time available for evaluation methodology. Independent work such as METR’s has filled part of the gap, and participants saw clear value in a research agenda and shared methods developed outside any single company. They also called for collaboration between academia and industry on benchmarks that measure agent capabilities more reliably.
From benchmark scores to real use
A recurring concern was the distance between benchmark scores and performance in real use. Open questions included:
- how to connect evaluation results to outcomes observed later
- when a score difference reflects a real difference in capability
- when LLM judges and rubrics can be trusted
- how much repetition long, open-ended tasks require
- how to verify agent reports against what occurred in the environment
- how to measure coordination among multiple agents
- how to validate simulated users
What participants asked for
Participants asked EvalScience to organize working groups on these questions, with remote talks open to researchers outside the Bay Area, and to provide a regular venue for sharing work in progress. Several offered to mentor early-career researchers. Compute and the recruitment of human participants were identified as shared constraints, and participants expect EvalScience to serve as a network for collaboration, hiring, and training.
Next steps
EvalScience will:
- draft a research agenda based on these discussions and share it with participants for comment ahead of the AI Agent Evaluation Science Symposium in New York on November 20
- launch working groups with monthly online talks
- match mentors with early-career researchers
- survey participants on compute and recruitment needs
- invite industry and academic researchers to scope a first joint benchmark project
Much remains to be built, and participation is welcome. We thank everyone who attended.