PulseAugur
中
实时 16:52:37
English(EN) QK-Wanda: Coupling Queries and Keys for Unstructured Pruning

新的QK-Wanda方法通过耦合查询和键来改进LLM剪枝

作者 PulseAugur 编辑部 · [1 个来源] ·

研究人员开发了QK-Wanda，一种用于剪枝大型语言模型的新颖方法，它改进了现有的Wanda技术。QK-Wanda将线性投影中的查询和键耦合起来，与Wanda相比，显著降低了重建误差。虽然QK-Wanda比Wanda稍慢，但它在Llama 2 70B等模型上展示了实质性的下游性能提升，改善了困惑度和零样本准确率。然而，QK-Wanda的有效性因模型而异，如在Llama 3.1 70B上所示，这表明局部重建误差并不总是整体模型质量的完美预测指标。 AI

影响 引入了一种更有效的剪枝技术，可能导致更高效的大型语言模型部署。

排序理由 该集群包含一篇详细介绍剪枝大型语言模型新方法的论文。[lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.LG 阅读 →

AI 生成摘要 · Google Gemini · 来自 1 个来源。 我们如何撰写摘要 →

新的QK-Wanda方法通过耦合查询和键来改进LLM剪枝

本文如何被排名

Signal score
4 / 100
Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket
Tool
该集群包含一篇详细介绍剪枝大型语言模型新方法的论文。[lever_c_demoted from research: ic=1 ai=1.0]
Source corroboration
Single-source cluster
Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics
paper, model release
Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance
High
Clearly on-topic for AI-industry coverage.
Story freshness
Same-day
Cluster formed today. Ranking reflects the current source set at time of score.

完整方法见我们的编辑标准。

报道来源 [1]

  1. arXiv cs.LG TIER_1 English(EN) · Ivan Ilin, Peter Richt\'arik ·

    QK-Wanda：耦合查询和键以实现非结构化剪枝

    arXiv:2610.01554v1 Announce Type: new Abstract: Wanda (Sun et al., 2024) prunes large language models by scoring weights independently within each linear projection, although queries and keys interact through dot products. We introduce QK-Wanda, which scores query and key weights…

相关实体

相关话题