English(EN) ARKD: Adaptive Reinforcement Learning-Guided Bidirectional KL Divergence Distillation for Text Generation

新的ARKD框架通过自适应KL散度增强LLM压缩

作者 PulseAugur 编辑部 · [2 个来源] · 2026-06-29 07:05

研究人员开发了ARKD，一个旨在提高大型语言模型（LLMs）压缩和性能的新型知识蒸馏框架。这种自适应强化学习引导的方法动态地权衡前向和反向KL散度目标，以更好地平衡主要分布拟合与长尾概率建模。实验表明，ARKD持续改进ROUGE L和BERTScore指标，优于现有方法。 AI

影响这项研究可能通过改进的压缩技术带来更高效、更强大的大型语言模型。

排序理由该集群包含一篇详细介绍LLM知识蒸馏新方法的学术论文。

AI 生成摘要 · Google Gemini · 来自 2 个来源。我们如何撰写摘要 →

报道来源 [2]

arXiv cs.AI TIER_1 English(EN) · Zilong Liu, Xuewen Zhang, Jinrui Xing, Juyi Qiao, Huiyong Wang, Junming Jiao · 2026-06-30 04:00

ARKD：用于文本生成的自适应强化学习引导的双向KL散度蒸馏

arXiv:2606.29869v1 Announce Type: cross Abstract: Knowledge distillation (KD) is a key technique for compressing Large Language Models (LLMs), yet methods relying on a single KL objective often fail to balance primary distribution fitting with long-tail probability modeling, limi…
arXiv cs.CL TIER_1 English(EN) · Junming Jiao · 2026-06-29 07:05

ARKD：用于文本生成的自适应强化学习引导的双向KL散度蒸馏

Knowledge distillation (KD) is a key technique for compressing Large Language Models (LLMs), yet methods relying on a single KL objective often fail to balance primary distribution fitting with long-tail probability modeling, limiting both generation quality and generalization. T…