研究人员开发了两种新方法，FOCUS和RePAIR，以解决剪枝大型语言模型（LLM）中的文本退化问题。LLM剪枝是一种压缩技术，可能会无意中增加重复输出和其他形式的退化，即使整体准确性指标看起来稳定。所提出的方法在令牌级别分析退化，将循环进入和持续作为关键因素。FOCUS旨在通过将蒸馏重新加权到高置信度区域来抑制泄漏，而RePAIR使用特定的连续对来鼓励多样化的输出并防止过早地陷入重复循环。实验表明，这两种技术都能有效减少重复，并提高开放式和基于指令的任务中生成文本的质量。 AI
影响 这些方法可以通过提高剪枝模型的质量、减少重复输出来实现LLM更有效的部署。
排序理由 该集群讨论了一篇关于改进剪枝大型语言模型的新研究论文。
AI 生成摘要 · Google Gemini · 来自 3 个来源。 我们如何撰写摘要 →
完整方法见我们的编辑标准。
arXiv:2608.28170v1 Announce Type: new Abstract: Purpose - This study investigates the feasibility of restoring missing text caused by physical lacunae in damaged ancient manuscripts using language models. Methodology - The study proposes different scenarios to replicate real-worl…
arXiv:2608.26676v1 Announce Type: cross Abstract: Pruning is a practical approach to compress large language models (LLMs), but it can amplify text degeneration, especially repetition loops, even when perplexity and task accuracy remain largely unchanged. In this work, we present…
Pruning is a practical approach to compress large language models (LLMs), but it can amplify text degeneration, especially repetition loops, even when perplexity and task accuracy remain largely unchanged. In this work, we present a token-level analysis of this failure mode by vi…