PulseAugur
中
实时 08:00:30
English(EN) On the Generalization Limitation of On-Policy Distillation for Post-RL Consolidation

新的蒸馏方法提高了LLM的推理能力，但泛化挑战依然存在 · 跟踪4个来源

作者 PulseAugur 编辑部 · [5 个来源] ·

研究人员正在探索策略内蒸馏（OPD）的新方法，这是一种用于改进大型语言模型的技术。一种方法vOPD引入了强化学习中的协方差基线，以稳定训练并减少梯度方差，实现了与计算成本更高的方法相当的性能。然而，另一项研究强调了OPD的泛化局限性，表明它会掩盖不完整的能力巩固，并导致在分布外任务上的性能下降。为了解决这些问题，正在开发EvoOPD等新策略来动态调整蒸馏过程中使用的特权信息，而仅顺序策略内蒸馏（O^3PD）则侧重于仅转移教师的令牌顺序而不是精确的概率，在推理和编码任务上表现出改进的性能。 AI

影响 蒸馏技术的这些进步可能带来更强大、更高效的LLM，但泛化方面的挑战需要进一步研究。

排序理由 arXiv上发表了多篇研究论文，详细介绍了LLM策略内蒸馏的新方法和局限性。

在 arXiv cs.LG 阅读 →

AI 生成摘要 · Google Gemini · 来自 5 个来源。 我们如何撰写摘要 →

新的蒸馏方法提高了LLM的推理能力，但泛化挑战依然存在 · 跟踪4个来源

本文如何被排名

Signal score
0 / 100
Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket
Research
arXiv上发表了多篇研究论文，详细介绍了LLM策略内蒸馏的新方法和局限性。
Source corroboration
5 independent sources
Strong cross-source corroboration — multiple independent publishers covered this within the clustering window.
Topics
paper, model release
Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance
High
Clearly on-topic for AI-industry coverage.
Story freshness
4 days old
Aged out of breaking-news scoring windows; ranking reflects the durable signal from the full source set.

完整方法见我们的编辑标准。

报道来源 [5]

  1. arXiv cs.AI TIER_1 English(EN) · Minjae Oh, Sangjun Song, Gyubin Choi, Yunho Choi, Yohan Jo ·

    KL for a KL: On-Policy Distillation with Control Variate Baseline

    arXiv:2605.07865v2 Announce Type: replace-cross Abstract: On-Policy Distillation (OPD) has emerged as a dominant post-training paradigm for Large Language Models, especially in reasoning domains. However, OPD remains unstable in practice due to the high gradient variance of its s…

  2. arXiv cs.LG TIER_1 English(EN) · Yichen Wang, Yufa Zhou, Siqi Zhu, Haozheng Luo, Jingzhi Bao, Zhichao Liu, Zijun Wei ·

    关于On-Policy Distillation用于RL后巩固的泛化限制

    arXiv:2611.05609v1 Announce Type: new Abstract: Multi-teacher on-policy distillation (MOPD) extends on-policy distillation (OPD) to consolidate capabilities from multiple teachers. Although prior work reports near-perfect in-distribution (IND) distillation performance, we find th…

  3. arXiv cs.LG TIER_1 English(EN) · Ximo Zhu, Ruiqi Liu, Yi Han, Zhengan Wu, Gengsheng Li, ZiAn Wang, Zhu Yuantian, Jin Jiang, Wenhao Wang ·

    EvoOPD：为黑盒on-policy蒸馏演进特权信息策略

    arXiv:2611.06542v1 Announce Type: new Abstract: Proprietary language models exhibit strong reasoning and coding capabilities, but their output distributions are inaccessible, making fine-grained distillation methods that rely on teacher logits difficult to apply directly to black…

  4. arXiv cs.CL TIER_1 English(EN) · Guangzeng Han, Yu Wu, Xiaolei Huang ·

    仅订单的策略内蒸馏

    arXiv:2611.02112v1 Announce Type: cross Abstract: On-policy distillation (OPD) trains a student on its own generations by matching the teacher's token probabilities. These probabilities entangle two kinds of information: which tokens the teacher prefers, and how confident it is. …

  5. Hugging Face Daily Papers TIER_1 English(EN) ·

    仅限指令的策略内蒸馏

    Order-only on-policy distillation trains students to match teacher token rankings rather than probabilities, improving reasoning performance without temperature tuning.

相关实体

相关话题