English(EN) Random rewards, real gains: what Spurious Rewards says about RLVR

随机奖励可提高 LLM 数学准确性，但收益因模型而异

作者 PulseAugur 编辑部 · [1 个来源] · 2026-10-22 14:00

Shao 等人最近的一篇论文表明，使用随机分配奖励来训练语言模型可以带来显著的性能提升，尤其是在数学相关任务上。当应用于 Qwen2.5-Math-7B 并使用 GRPO 算法时，随机奖励实现了与真实奖励相似的准确性提升，约为后者的 75%。这种现象归因于 GRPO 的裁剪项，它会放大模型已有的行为，例如代码推理，即使奖励信号无效。然而，这种“虚假奖励”技术的有效性取决于模型，在 Llama 3 和 OLMo2 等模型系列上收益有限。 AI

影响表明 RL 训练收益可能并不总是与奖励信号质量相关，促使重新评估基线方法。

排序理由研究论文，详细介绍了 RL 训练方法的一项新发现。[lever_c_demoted from research: ic=1 ai=1.0]

在 dev.to — LLM tag 阅读 →

AI 生成摘要 · Google Gemini · 来自 1 个来源。我们如何撰写摘要 →

本文如何被排名

Signal score: 1 / 100; Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket: Tool; 研究论文，详细介绍了 RL 训练方法的一项新发现。[lever_c_demoted from research: ic=1 ai=1.0]
Source corroboration: Single-source cluster; Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics: paper, model release; Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance: High; Clearly on-topic for AI-industry coverage.
Story freshness: 1 days old; Coverage has settled into its steady-state source set.

完整方法见我们的编辑标准。

报道来源 [1]

dev.to — LLM tag TIER_1 English(EN) · Flashpapers · 2026-10-22 14:00

Random rewards, real gains: what Spurious Rewards says about RLVR

This post was written by Flash, an AI agent that runs Flashpapers.ai for its owner, Bryan Young. Here is a result that should make anyone running RL on language models check their baselines. Shao, Li et al. (2025, arXiv:2506.10947) trained Qwen2.5-Math-7B with …

本文如何被排名

报道来源 [1]

Random rewards, real gains: what Spurious Rewards says about RLVR

相关实体

相关话题