English(EN)StepX-OPD: Rethinking Multi-Teacher On-Policy Distillation in Diffusion Models
新的蒸馏方法通过调整教师和 token 来增强 LLM 推理能力
作者PulseAugur 编辑部·[7 个来源]·
研究人员正在开发先进的策略内蒸馏技术，以提高大型语言模型的性能。一种方法是学生前缀教师适应（SPTA），它修改教师模型以更好地处理学生生成的前缀，从而提高数学推理任务的指导可靠性。另一种方法是教师演化自蒸馏（TESD），它通过使用任务正确性和上下文独立性的奖励来提高教师的自学能力，同时还自适应地降低不可靠指导的权重。第三种技术是多教师自适应策略内自蒸馏（MTA-OPSD），它更进一步，允许每个 token 从池中选择其首选教师，在数学推理基准测试中显示出显著的收益。
AI
影响
这些先进的蒸馏技术旨在提高 LLM 的推理能力，有可能为复杂任务带来更可靠、更准确的 AI 系统。
arXiv:2611.07722v1 Announce Type: new Abstract: On-policy distillation (OPD) improves language-model reasoning through dense teacher feedback on student-generated responses.We first observe an amplitude disparity in this feedback: on math reasoning responses, style-related positi…
arXiv:2611.03133v1 Announce Type: new Abstract: On-policy distillation (OPD) mitigates the student's exposure bias by optimizing on student-generated trajectories. However, student-generated prefixes may contain reasoning errors or follow patterns unfamiliar to the teacher, under…
A deployable text-to-image diffusion model must hold several capabilities at once, such as instruction following, text rendering and human preference, and must sample in a few steps. Capabilities and few-step sampling are built in separate stages: a full-step model first consolid…
StepX-OPD jointly trains a text-to-image diffusion model to acquire capabilities and few-step sampling simultaneously, outperforming sequential approaches while reducing training cost.
arXiv:2610.38637v1 Announce Type: cross Abstract: On-Policy Self-Distillation (OPSD), which aligns a student model's token-level distribution with that of its self-teacher prompted by privileged context, has emerged as a promising direction to mitigate reward sparsity in sequence…
Multi-teacher Adaptive On-policy Self-Distillation selects a preferred teacher for each token during training, improving mathematical reasoning performance over fixed-teacher methods.
Teacher-Evolution Self-Distillation improves on-policy self-distillation by continuously enhancing self-teaching reliability through reward-guided teacher optimization and adaptive down-weighting of unreliable samples.
