新的 MUDDLE 基准测试 LLM 对抗干扰因素和长度影响的文档理解能力

作者 PulseAugur 编辑部 · [1 个来源] · 2026-09-01 04:00

研究人员推出 MUDDLE，这是一个新的基准测试，旨在通过区分文档长度和干扰信息的影响来评估文档问答系统。该基准测试使用了 270 个经过人工标注的问题，每个问题在五种条件下进行测试：仅源文档、源文档加相似干扰项、源文档加随机干扰项。对 GPT-5 mini 的初步测试表明，主题相似的干扰项比等长随机干扰项对准确率的负面影响更大。 AI

影响该基准测试通过突出干扰信息的影响，有望促使更强大的文档 QA 系统。

排序理由该集群描述了一个用于评估 LLM 能力的新学术基准测试。[lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.AI 阅读 →

AI 生成摘要 · Google Gemini · 来自 1 个来源。我们如何撰写摘要 →

本文如何被排名

Signal score: 22 / 100; Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket: Tool; 该集群描述了一个用于评估 LLM 能力的新学术基准测试。[lever_c_demoted from research: ic=1 ai=1.0]
Source corroboration: Single-source cluster; Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics: paper, product; Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance: High; Clearly on-topic for AI-industry coverage.
Story freshness: Breaking (< 6h); Fresh story with cross-source coverage still developing. Ranking may shift as more sources report.

完整方法见我们的编辑标准。

报道来源 [1]

arXiv cs.AI TIER_1 English(EN) · Jason Luo, Saibilila Abudukelimu, Judy Song, Andrew Feng, Shivank Garg, Vasu Sharma, Kevin Zhu · 2026-09-01 04:00

MUDDLE：衡量在干扰和长度效应下文档的理解能力

arXiv:2608.29477v1 Announce Type: cross Abstract: Document question-answering systems increasingly answer questions over collections of retrieved documents rather than one clean source, so robustness to distracting context matters as much as reading ability. When such systems fai…

本文如何被排名

报道来源 [1]

MUDDLE：衡量在干扰和长度效应下文档的理解能力

相关实体

相关话题