研究人员开发了一种名为排序分组策略优化 (sGPO) 的新训练策略，以提高具有可验证奖励的强化学习 (RLVR) 的效率。该方法使用少量的推理计算来识别查询难度，从而更好地分配训练资源。通过分析查询并调整训练组大小，sGPO 可显著减少计算浪费，并将总训练计算量最多降低三倍，同时保持或提高性能。 AI
影响 降低了 RLVR 的训练计算量，有可能加速需要可验证奖励的领域的研究和开发。
排序理由 该集群包含一篇详细介绍新研究方法的学术论文。
AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →
arXiv:2606.08854v1 Announce Type: cross Abstract: Standard Reinforcement Learning with Verifiable Rewards (RLVR) training allocates a fixed rollout budget to every query, without regard for what each query's difficulty means for the current policy. This leads to two symmetric fai…
Standard Reinforcement Learning with Verifiable Rewards (RLVR) training allocates a fixed rollout budget to every query, without regard for what each query's difficulty means for the current policy. This leads to two symmetric failure modes: easy queries produce near-zero advanta…