Did agents fully reproduce most PAPERBENCHX tasks?
No. The authors report that the best configuration fully reproduced 14% of tasks under their 90% score threshold. Mean partial scores were higher, showing that progress on individual steps often did not compose into a complete reproduction.
Answered in
PAPERBENCHX Tests Whether AI Can Rebuild Evidence, Not Just Explain a PaperA 93-task benchmark across scientific fields finds a large gap between AI agents running parts of an experiment and reproducing its evidence.
Read the full analysisOther questions this article answers
More ai science questions
Every answer on Crashtech is written by the editor of the article it comes from — never auto-summarised. Browse all answers or the Ai Science beat.