arXiv 2610.21767
From Research Goals to Scientific Closure: Evaluating Agents Through Evidence-Supported Physics Research
AI30 upvotes
Authors: Yijia Mao, Song Dai, Tao Chen, Xiangyu Lu, Linqian Zeng, Chenxi Hu, Sitong Zhou, Dongyang He +9
Overview
The authors introduce PhysClosureBench, a benchmark of 50 expert-verified, goal-specified physics research tasks in strong-field, attosecond, and neighboring areas, where agents must organize investigations and generate and validate their own evidence. Performance is measured by a 20-criterion Atomic Score for local accomplishment and by Research Closure levels RC0–RC4 for whether indispensable evidence and applicable scope conditions are satisfied. Seven frontier LLMs are evaluated on the benchmark.
Introduction
Useful scientific agents must do more than complete isolated steps correctly; they must organize investigations that reach well-supported conclusions. Measuring this requires assessing whether a research question is actually resolved, not just whether individual actions are accurate.
Results
Claude-Opus-5 achieves the highest mean Atomic Score (6.29/10) but reaches RC4 in only 32.7% of attempts. Even among attempts scoring at least 8/10, 36.2% still fall short of RC4.
Conclusion
The results reveal a persistent gap between AI agents' local scientific accomplishments and their ability to achieve complete research closure.