English(EN) EarlySelectKV: Anticipatory KV Selection via Cross-Layer Query Proxies

EarlySelectKV 优化长上下文 LLM 推理，提高速度并减少内存使用

作者 PulseAugur 编辑部 · [1 个来源] · 2026-11-04 05:00

研究人员开发了 EarlySelectKV，一种优化大型语言模型长上下文推理的新方法。该技术使用来自早期隐藏状态的代理查询来预计算选定的 KV 索引，从而实现更快的路由和更少的内存使用。在 LongBench 和 RULER 等基准测试中，使用 RocketKV、Quest 和 Loki 等 KV 缓存进行的评估表明，吞吐量提高了 12.4%，GPU 内存减少了 11.1%。 AI

影响提高了长上下文 LLM 推理的效率，可能支持更复杂的应用。

排序理由这是一篇详细介绍 LLM 推理优化新方法的论文。[lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.CL 阅读 →

AI 生成摘要 · Google Gemini · 来自 1 个来源。我们如何撰写摘要 →

本文如何被排名

Signal score: 42 / 100; Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket: Tool; 这是一篇详细介绍 LLM 推理优化新方法的论文。[lever_c_demoted from research: ic=1 ai=1.0]
Source corroboration: Single-source cluster; Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics: paper, infra; Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance: High; Clearly on-topic for AI-industry coverage.
Story freshness: Breaking (< 6h); Fresh story with cross-source coverage still developing. Ranking may shift as more sources report.

完整方法见我们的编辑标准。

报道来源 [1]

arXiv cs.CL TIER_1 English(EN) · Yuhao Liu, Yuehao Li, Linghe Zhang, Gaobin Huang, Junran Li, Hongchang Wang, Haifang Jian · 2026-11-04 05:00

EarlySelectKV：通过跨层查询代理进行预期KV选择

arXiv:2611.01715v1 Announce Type: new Abstract: Query-conditioned sparse KV selection reduces long-context inference cost by attending to a subset of historical KV entries at each decoding step. Existing selectors must wait for the target-layer query before scoring candidates and…

本文如何被排名

报道来源 [1]

EarlySelectKV：通过跨层查询代理进行预期KV选择

相关实体

相关话题