研究人员推出了一种新颖的蒸馏框架 Budgeted LoRA，旨在创建更高效的用于推理的大型语言模型。该方法将模型压缩视为一个结构化计算分配问题，允许根据全局计算预算在密集和低秩路径之间重新分配容量。该方法能够控制推理速度提升，实证结果表明在激进预算下可实现显著的速度提升，同时在某些任务上保持具有竞争力的准确性。 AI
影响 引入了一种优化 LLM 推理效率的新方法，有可能降低部署的计算成本。
排序理由 这是一篇详细介绍模型蒸馏和效率新方法的 ist 研究论文。
AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →
arXiv:2605.04341v1 Announce Type: new Abstract: We study distillation for large language models under explicit compute constraints, with the goal of producing student models that are not only cheaper to train, but structurally efficient at inference time. While prior approaches t…
We study distillation for large language models under explicit compute constraints, with the goal of producing student models that are not only cheaper to train, but structurally efficient at inference time. While prior approaches to parameter-efficient distillation, such as LoRA…