English(EN) What Matters for Aggressive Decoding-Time KV Eviction? Temporal Aggregation and Ranking Preservation

KV 缓存压缩研究确定时间聚合为关键因素

作者 PulseAugur 编辑部 · [1 个来源] · 2026-09-04 04:00

研究人员调查了时间聚合和排名保留对大型语言模型解码时 KV 缓存压缩的影响。他们发现指数移动平均 (EMA) 聚合可以使评分器修改在驱逐级别无法区分，从而实现稳定的保留集。这促成了 InertiaKV 和 InertiaKV-Lazy 的开发，这些方法与完全刷新方法相比，将解码吞吐量提高了 1.34-1.46 倍。该研究还探讨了无评分解码，它在显著降低计算成本的同时保持了平均质量。 AI

影响引入了提高 LLM 解码吞吐量和降低计算成本的方法。

排序理由学术论文，详细介绍了 LLM 中 KV 缓存压缩的新颖方法。[lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.AI 阅读 →

AI 生成摘要 · Google Gemini · 来自 1 个来源。我们如何撰写摘要 →

本文如何被排名

Signal score: 17 / 100; Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket: Tool; 学术论文，详细介绍了 LLM 中 KV 缓存压缩的新颖方法。[lever_c_demoted from research: ic=1 ai=1.0]
Source corroboration: Single-source cluster; Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics: paper, infra; Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance: High; Clearly on-topic for AI-industry coverage.
Story freshness: Breaking (< 6h); Fresh story with cross-source coverage still developing. Ranking may shift as more sources report.

完整方法见我们的编辑标准。

报道来源 [1]

arXiv cs.AI TIER_1 English(EN) · Bo Zeng, Yu Zhao, Yefeng Liu, Zhihong Lu, Xuanfan Ni, Xintong Wang · 2026-09-04 04:00

激进解码时 KV 驱逐的关键是什么？时间聚合与排名保留

arXiv:2609.03515v1 Announce Type: new Abstract: Decoding-time KV cache compression research focuses heavily on designing better token scoring functions, while the temporal rule that aggregates scores across decode steps is often treated as an implementation detail. Under aggressi…

本文如何被排名

报道来源 [1]

激进解码时 KV 驱逐的关键是什么？时间聚合与排名保留

相关实体

相关话题