新的自蒸馏方法增强了大型语言模型的推理能力和训练稳定性

报道来源 [3]

1. arXiv cs.LG TIER_1 English(EN) · Akiyoshi Tomihari, Issei Sato · 2026-05-07 04:00

   Power Distribution 桥接采样、自奖励 RL 和自蒸馏

   arXiv:2605.04542v1 Announce Type: new Abstract: Recent analyses question whether reinforcement learning (RL) is responsible for strong reasoning in large language models (LLMs). At the same time, distillation and inference-time sampling, including power sampling, have emerged as …

2. arXiv cs.LG TIER_1 English(EN) · Xin Yu, Liuchen Liao, Yiwen Zhang, Yingchen Yu, Lingzhou Xue, Qinzhen Guo · 2026-05-07 04:00

   基于偏好的自蒸馏：通过奖励正则化超越KL匹配

   arXiv:2605.05040v1 Announce Type: new Abstract: On-policy distillation is an efficient alternative to reinforcement learning, offering dense token-level training signals. However, its reliance on a stronger external teacher has driven recent work on on-policy self-distillation, w…

3. arXiv cs.AI TIER_1 English(EN) · Qinzhen Guo · 2026-05-06 15:31

   基于偏好的自蒸馏：通过奖励正则化超越KL匹配

   On-policy distillation is an efficient alternative to reinforcement learning, offering dense token-level training signals. However, its reliance on a stronger external teacher has driven recent work on on-policy self-distillation, where the same model serves as both teacher and s…