RLVR在LLM中的收益可能源于答案格式，而非能力提升

作者 PulseAugur 编辑部 · [1 个来源] · 2026-10-16 04:00

一篇新论文对强化学习可验证奖励（RLVR）在大语言模型中报告的收益提出了质疑，认为在MMLU等域外任务上的改进很大程度上归因于答案格式化，而非真正的能力增强。研究人员发现，当答案提取方法调整为在文本任意位置查找答案，而不只是在特定框内查找时，报告的收益显著减少。这表明RLVR过程可能主要是在教授模型将答案置于提取器偏好的格式中，而不是提高其根本的推理或知识能力。 AI

影响表明当前的RLVR收益可能很肤浅，促使重新评估LLM的训练和评估方法。

排序理由该集群包含一篇详细介绍LLM训练方法研究发现的学术论文。[lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.CL 阅读 →

AI 生成摘要 · Google Gemini · 来自 1 个来源。我们如何撰写摘要 →

本文如何被排名

Signal score: 17 / 100; Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket: Tool; 该集群包含一篇详细介绍LLM训练方法研究发现的学术论文。[lever_c_demoted from research: ic=1 ai=1.0]
Source corroboration: Single-source cluster; Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics: paper, model release; Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance: High; Clearly on-topic for AI-industry coverage.
Story freshness: Breaking (< 6h); Fresh story with cross-source coverage still developing. Ranking may shift as more sources report.

完整方法见我们的编辑标准。

报道来源 [1]

arXiv cs.CL TIER_1 English(EN) · Zongye Lyu · 2026-10-16 04:00

答案相同，包装新颖：RLHF 的域外收益有多少来自答案格式？

arXiv:2610.20639v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards (RLVR) is credited with gains beyond the domain its rewards check, such as higher MMLU accuracy after training on mathematics. Such accuracies are usually read out of the model's text…

本文如何被排名

报道来源 [1]

答案相同，包装新颖：RLHF 的域外收益有多少来自答案格式？

相关实体

相关话题