Contributed research
Selected papers, extended abstracts, and posters presenting ongoing and published work in agent evaluation.
Fall 2026 · New York City
Agent Evaluation Science
Symposium
A one-day symposium on the science and practice of evaluating agentic systems.
- Date
- Friday, November 20, 2026
- Location
- Downtown Manhattan · New York City
Registration is open to the public; acceptance prioritizes presenting authors.
Explore the symposiumScientific evaluation◆Construct validity◆Measurement reliability◆Benchmark integrity◆Agent safety◆Grader design◆Failure analysis◆Evaluation infrastructure◆Real-world evidence◆Reproducible systems◆
Symposium program
Bringing evaluation questions into one room
Agent evaluation is developing across research, industry, and different application domains, often around separate methods and problems. The symposium brings these perspectives together to compare evidence, surface disagreements, and identify priorities for future evaluation research.
Evaluation science agenda · Program format
0102 0304
Talks & fireside
Invited researchers and practitioners will share current work, open problems, and perspectives on where agent evaluation is heading.
View speakers →Evaluation in practice
Cases and lessons from evaluating agents in real-world settings and workflows.
View program →Agenda discussion
A discussion of open questions and priorities that could shape future evaluation research.
Summary available after the symposiumCall for presentations
Call for presentations · Submissions closed
Submissions closed on October 25 and authors hear back on November 5. The call covered evaluation studies, audits, negative results, failure analyses, tools, datasets, production lessons, and real-world evaluation cases, with an extended abstract track for ongoing and previously published work.
- Evaluation studies
- Audits and replications
- Negative results
- Failure analyses
- Tools and datasets
- Production lessons
- Real-world evaluation cases
Industry and real-world submissions are assessed on documented evidence, production lessons, and findings relevant beyond a single system or setting. A new benchmark or algorithm is not required.
Submissions open
August 252026Full submission deadline
October 252026Notification of acceptance
November 52026Symposium
November 202026Affiliations represented
Speaker and committee affiliations
- EvalScience
- Harvard University
- Massachusetts General Hospital
- BRIDGE GenAI Lab
- Princeton University
- Cornell University · Cornell Tech
- The University of Texas at Austin
- University of Technology Sydney
- Australian Artificial Intelligence Institute
- University of Alberta
- Amii
- Meta Superintelligence Labs
- Sony AI
- Raycaster
- Google DeepMind
- Massachusetts Institute of Technology
- MIT Media Lab
- University of Massachusetts Amherst
- Max Planck Institute for Intelligent Systems
- Columbia University
- DAPLab
- New York University
- Cohere Labs
- J.P. Morgan
- IBM Research
- Microsoft Research
- University of Illinois Urbana-Champaign
- Dartmouth College
- Vector Institute
Affiliations describe speakers’ and committee members’ institutional or organizational connections. They do not indicate sponsorship.
Agent Evaluation Science Fall 2026Friday, November 20 · Downtown Manhattan, New York City
Meet the committee
