English(EN) LLM Benchmark Datasets Should Be Contamination-Resistant

论文呼吁开发抗预训练数据污染的大语言模型基准

作者 PulseAugur 编辑部 · [1 个来源] · 2026-05-19 15:33

一篇新论文认为，用于评估大语言模型（LLM）的基准数据集必须能够抵抗预训练数据的污染。作者们指出，许多现有基准已包含在 LLM 的训练语料库中，这削弱了它们衡量真正泛化能力的有效性。他们提议利用 Transformer 模型中的架构不对称性来创建在训练期间无法学习但在推理时仍可用的数据集，并呼吁社区采纳这些抗污染方法。 AI

影响通过防止基准污染，确保对 LLM 能力进行更可靠的评估。

排序理由该集群包含一篇提出 LLM 评估新方法的学术论文。[lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.AI 阅读 →

AI 生成摘要 · Google Gemini · 来自 1 个来源。我们如何撰写摘要 →

本文如何被排名

Signal score: 0 / 100; Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket: Tool; 该集群包含一篇提出 LLM 评估新方法的学术论文。[lever_c_demoted from research: ic=1 ai=1.0]
Source corroboration: Single-source cluster; Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics: paper, safety; Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance: High; Clearly on-topic for AI-industry coverage.
Story freshness: 113 days old; Aged out of breaking-news scoring windows; ranking reflects the durable signal from the full source set.

完整方法见我们的编辑标准。

报道来源 [1]

arXiv cs.AI TIER_1 English(EN) · Suhang Wang · 2026-05-19 15:33

LLM基准数据集应具有抗污染性

Benchmark datasets are critical for reproducible, reliable, and discriminative evaluation of LLMs. However, recent studies reveal that many benchmark datasets are included in pretraining corpora, i.e., $\textit{contaminated}$, which diminishes their value as reliable measures of …

本文如何被排名

报道来源 [1]

LLM基准数据集应具有抗污染性

相关实体

相关话题