研究人员开发了 GAMUT，这是一个旨在评估长文本 AI 生成内容事实完整性的新基准。与以往关注单个声明准确性的方法不同，GAMUT 评估响应是否包含所有必要信息，解决了“事实性的缺失一半”。该基准利用了一个两级元评分标准系统，该系统可以机械地编译成机器可评分的清单，即使使用 LLM 裁判也证明是有效的。在评估中，GAMUT 对 14 个前沿和开源模型提出了重大挑战，谷歌的 Gemini 3.1 Pro 取得了 58.7% 的最高分。 AI
影响 该基准可以推动 AI 在生成全面且事实完整的长文本内容方面的能力得到改进。
排序理由 该集群描述了一篇介绍用于评估 AI 模型输出的基准的新学术论文。
AI 生成摘要 · Google Gemini · 来自 3 个来源。 我们如何撰写摘要 →
完整方法见我们的编辑标准。
arXiv:2607.19322v1 Announce Type: new Abstract: Evaluating the factuality of long-form generations has focused predominantly on precision, measuring whether the claims a model makes are correct. The dominant decompose-search-verify pipeline catches incorrect claims well but says …
Evaluating the factuality of long-form generations has focused predominantly on precision, measuring whether the claims a model makes are correct. The dominant decompose-search-verify pipeline catches incorrect claims well but says little about whether a response contains all the…
<h2> What Changed </h2> <p>For the past several years, the evaluation of large language models (LLMs) has been heavily skewed toward precision. Developers and researchers have relied on the 'decompose-search-verify' pipeline, a methodology that excels at identifying incorrect cla…