研究人员开发了新的策略内蒸馏技术，以改进多模态AI模型。OPOD方法将学生响应路由到特定模态的教师，在各种基准测试中取得了最先进的成果。对比策略内蒸馏（COPD）和策略内增量蒸馏（OPD^2）分别通过关注相对推理兼容性和指令调优的增量信号，进一步改进了这一点，从而实现了更高效、更强大的模型。 AI
影响 这些蒸馏技术有望实现更高效、更强大的多模态AI模型，从而加速其在复杂推理任务中的应用。
排序理由 该集群包含多篇详细介绍AI模型蒸馏新方法的学术论文。
在 Hugging Face Daily Papers 阅读 →
AI 生成摘要 · Google Gemini · 来自 5 个来源。 我们如何撰写摘要 →
完整方法见我们的编辑标准。
arXiv:2607.20918v1 Announce Type: new Abstract: Omni-modal models can handle text, images, and audio in one system, but improving all of these abilities together remains difficult. Training a single model on pooled multimodal data often fails to match models specialized for indiv…
On-policy Distillation (OPD) supervises a student model on trajectories sampled from its own policy by minimizing the divergence between the output distributions of the teacher and student at each token position, thereby providing dense token-level supervision. Although existing …
On-policy distillation is an alternative post-training method in reinforcement learning that alleviates the constraints imposed by reward models by providing token-level supervision from a teacher model. Although on-policy distillation has been studied and applied across various …
We study on-policy distillation (OPD) for agentic tasks, where an LLM agent interacts with an environment over multiple turns and a student imitates a teacher over these multi-turn interaction histories. Fully online OPD is costly because each update requires fresh student rollou…
arXiv:2607.19046v1 Announce Type: new Abstract: On-policy Distillation (OPD) supervises a student model on trajectories sampled from its own policy by minimizing the divergence between the output distributions of the teacher and student at each token position, thereby providing d…