研究人员引入了一种名为On-Policy Delta Distillation (OPD^2) 的新方法，以提高大型语言模型中推理能力的迁移。该技术利用“delta信号”（表示指令微调前教师模型与其基础模型之间的差异）来提供更直接的监督。在数学、科学和代码推理任务上的实验表明，OPD^2 的性能显著优于传统的on-policy distillation，使得LLM能够以最小的训练后调整实现强大的推理性能。 AI
影响 增强了LLM的推理迁移能力，有望为复杂任务带来更强大、更高效的模型。
排序理由 该集群包含一篇详细介绍改进LLM推理能力新方法的论文。
AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →
完整方法见我们的编辑标准。
arXiv:2607.15161v1 Announce Type: cross Abstract: On-policy distillation is an alternative post-training method in reinforcement learning that alleviates the constraints imposed by reward models by providing token-level supervision from a teacher model. Although on-policy distill…
On-policy distillation is an alternative post-training method in reinforcement learning that alleviates the constraints imposed by reward models by providing token-level supervision from a teacher model. Although on-policy distillation has been studied and applied across various …