新的SCI-AGENT-EXAM基准揭示了LLM代理在科学研究中的性能差距

作者 PulseAugur 编辑部 · [1 个来源] · 2026-10-26 04:00

一个名为SCI-AGENT-EXAM的新基准已被开发出来，用于系统地评估大型语言模型（LLM）代理在科学研究工作流中的能力。该基准包含11个科学领域的120个任务和7种研究任务类型，需要可执行代码和文本输出。使用10个前沿LLM代理进行的实验表明，GPT-6 Astra取得了最高分80.9%，但只有26%的运行达到了0.95的成功阈值，这凸显了执行效率方面显著的性能差距和差异。 AI

影响该基准将能够更严格地评估LLM代理执行复杂科学任务的能力，有可能加速开发更强大的AI研究助手。

排序理由该集群描述了一篇介绍LLM代理基准的新学术论文。[lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.CL 阅读 →

AI 生成摘要 · Google Gemini · 来自 1 个来源。我们如何撰写摘要 →

本文如何被排名

Signal score: 15 / 100; Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket: Tool; 该集群描述了一篇介绍LLM代理基准的新学术论文。[lever_c_demoted from research: ic=1 ai=1.0]
Source corroboration: Single-source cluster; Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics: paper, product; Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance: High; Clearly on-topic for AI-industry coverage.
Story freshness: Breaking (< 6h); Fresh story with cross-source coverage still developing. Ranking may shift as more sources report.

完整方法见我们的编辑标准。

报道来源 [1]

arXiv cs.CL TIER_1 English(EN) · Tianyun Zhong, Wei Wang, Jialin Li, Jianhong Tu, Liyao Li, Chuqiao Kuang, Tianshi Zheng, Songchi Zhou, Weiqi Zhai, Qile Jin, Jiajia Li, Jiasheng Zheng, Boxi Cao, Yaojie Lu, Hongyu Lin, Shiwei Ye, Bing Zhao, Hu Wei, Lin Qu, Xianpei Han · 2026-10-26 04:00

SCI-AGENT-EXAM: 科学研究任务中的智能体基准测试

arXiv:2610.32441v1 Announce Type: new Abstract: Large language model (LLM) agents are increasingly capable of supporting scientific workflows, but their capabilities across diverse research activities remain difficult to assess systematically. Existing benchmarks often focus on s…

本文如何被排名

报道来源 [1]

SCI-AGENT-EXAM: 科学研究任务中的智能体基准测试

相关实体

相关话题