两篇新研究论文提出了一种在涉及多个奖励时，更有效地将信用分配给语言模型输出中特定代币的方法。第一篇论文MRTPO利用验证器信息，在不使用辅助模型的情况下，将奖励优势直接分配给相关代币。第二篇论文P2T采用一种无训练的、基于梯度的方法，通过估计其影响来将整体奖励归因于单个代币，仅需一次前向和后向传播。 AI
影响 这些方法可能导致语言模型训练效率的提高，尤其是在需要多重标准的复杂任务中。
排序理由 arXiv上发表的两篇学术论文，详细介绍了语言模型强化学习中代币级信用分配的新方法。
AI 生成摘要 · Google Gemini · 来自 3 个来源。 我们如何撰写摘要 →
完整方法见我们的编辑标准。
arXiv:2610.21439v1 Announce Type: new Abstract: Language models are increasingly trained against several verifiable rewards at once, such as the correctness and the format of a tool call, or the individual constraints of a complex instruction. Reward-decoupled normalization (GDPO…
MRTPO improves multi-reward training by assigning each reward's advantage only to relevant tokens via replay-based relevance measurement, accelerating convergence and boosting performance.
arXiv:2610.18704v1 Announce Type: new Abstract: In this paper, we propose an extremely efficient, training-free method to extract token-level reward signals directly from an existing deep reward model. Our core idea is to attribute the overall process reward to individual tokens …