English(EN) Benchmarks as Software: Terminal-Bench and the Case for Continuous Benchmarks

新基准测试解决了过时的 AI 评估和资源分配影响问题

作者 PulseAugur 编辑部 · [1 个来源] · 2026-11-06 05:00

一篇新论文介绍了 Terminal-Bench 2.1，这是一个用于评估编码代理的更新基准测试，解决了“基准测试维护债务”问题，即随着模型改进，旧基准测试会过时。研究人员发现，原始 Terminal-Bench 2.0 中约有 30% 存在问题，而 2.1 版本中的修复将准确率提高了 10 多个百分点。研究还强调了资源分配（如时间限制）对代理性能的重大影响，表明这些参数是基准测试规范的关键组成部分。 AI

影响强调了持续评估 AI 模型的需求以及资源分配对性能指标的影响。

排序理由学术论文，介绍新的基准测试和方法论。[lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.AI 阅读 →

AI 生成摘要 · Google Gemini · 来自 1 个来源。我们如何撰写摘要 →

本文如何被排名

Signal score: 11 / 100; Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket: Tool; 学术论文，介绍新的基准测试和方法论。[lever_c_demoted from research: ic=1 ai=1.0]
Source corroboration: Single-source cluster; Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics: paper, product; Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance: High; Clearly on-topic for AI-industry coverage.
Story freshness: Same-day; Cluster formed today. Ranking reflects the current source set at time of score.

完整方法见我们的编辑标准。

报道来源 [1]

arXiv cs.AI TIER_1 English(EN) · E. Kelly Buchanan, Crystal Zhou, Scott W. Linderman, Christopher R\'e · 2026-11-06 05:00

基准测试即软件：Terminal-Bench 与持续基准测试的案例

arXiv:2611.08417v1 Announce Type: new Abstract: Coding agents are among the most economically consequential deployments of LLMs to date, making reliable evaluation increasingly important. Yet even benchmarks that are carefully validated at release can become stale. As models impr…

本文如何被排名

报道来源 [1]

基准测试即软件：Terminal-Bench 与持续基准测试的案例

相关实体

相关话题