English(EN) RoPE-Aware Bit Allocation for KV-Cache Quantization

新方法通过KV缓存压缩和量化提升LLM效率

作者 PulseAugur 编辑部 · [8 个来源] · 2026-06-21 08:48

研究人员开发了新的方法来提高大型语言模型（LLMs）的效率，方法是压缩它们的键值（KV）缓存。一种名为InfoKV的方法，利用预测不确定性等信息论信号以及注意力权重，来更好地估计token重要性以进行压缩，在Llama-3.1和DeepSeek-R1等模型的长上下文推理任务上表现出改进的性能。另一种方法Block-GTQ，专注于RoPE感知比特分配用于KV缓存量化，根据RoPE中不同频率块对量化误差的敏感度来调整比特分布。该技术显著提高了长上下文检索和推理等任务的下游性能，并能在最小的质量损失下实现大量的KV缓存压缩，如在Llama-3.1-8B-Instruct和Qwen2.5-3B-Instruct等模型上所展示的。 AI

影响 KV缓存压缩和量化方面的这些进展有望显著降低LLMs的内存使用量并提高推理速度，从而实现更长的上下文窗口和更高效的部署。

排序理由多篇研究论文和社区讨论详细介绍了LLMs中KV缓存压缩和量化的新颖方法。

在 Hugging Face Daily Papers 阅读 →

AI 生成摘要 · Google Gemini · 来自 8 个来源。我们如何撰写摘要 →

报道来源 [8]

arXiv cs.AI TIER_1 English(EN) · Jushi Kai, Zhuiri Xiao, Alexandra Birch, Zhouhan Lin · 2026-06-26 04:00

Information-Aware KV Cache Compression for Long Reasoning

arXiv:2606.26875v1 Announce Type: cross Abstract: Reasoning capability has advanced rapidly in large language models (LLMs), leading to an increasing size of key-value (KV) cache in both prefilling and decoding stages. Existing KV cache compression methods mainly rely on attentio…
arXiv cs.AI TIER_1 English(EN) · Zhouhan Lin · 2026-06-25 11:03

Information-Aware KV Cache Compression for Long Reasoning

Reasoning capability has advanced rapidly in large language models (LLMs), leading to an increasing size of key-value (KV) cache in both prefilling and decoding stages. Existing KV cache compression methods mainly rely on attention weights to estimate token importance. While atte…
Hugging Face Daily Papers TIER_1 English(EN) · 2026-06-25 00:00

Information-Aware KV Cache Compression for Long Reasoning

InfoKV is an entropy-aware KV cache compression framework that enhances long-context reasoning in LLMs by incorporating information-theoretic signals alongside attention weights.
arXiv cs.CL TIER_1 English(EN) · Fengfeng Liang, Yuechen Zhang, Jiaya Jia · 2026-06-24 04:00

RoPE-Aware Bit Allocation for KV-Cache Quantization

arXiv:2606.24033v1 Announce Type: cross Abstract: Existing low-bit KV-cache quantizers often treat each cached key as a flat vector. Under RoPE, however, a key's contribution to a future attention logit decomposes into a position-dependent sum over two-dimensional frequency block…
arXiv cs.CL TIER_1 English(EN) · Jiaya Jia · 2026-06-23 00:17

RoPE-Aware Bit Allocation for KV-Cache Quantization

Existing low-bit KV-cache quantizers often treat each cached key as a flat vector. Under RoPE, however, a key's contribution to a future attention logit decomposes into a position-dependent sum over two-dimensional frequency blocks. This makes key-cache quantization a block-wise …
Hugging Face Daily Papers TIER_1 English(EN) · 2026-06-23 00:00

RoPE-Aware Bit Allocation for KV-Cache Quantization

Block-GTQ introduces a RoPE-aware bit allocation method for key-cache quantization that improves attention accuracy and downstream performance through adaptive bit distribution and packed cache serving.
r/LocalLLaMA TIER_1 English(EN) · /u/crusaderky · 2026-06-23 15:12

我绘制了 Qwen3.6-35B-A3B 和 Gemma4-E2B QAT 的 KV 缓存量化的 KLD 图

<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1udjvhd/i_mapped_the_kld_of_kv_cache_quantization_for/"> <img alt="I mapped the KLD of KV cache quantization for Qwen3.6-35B-A3B and Gemma4-E2B QAT" src="https://preview.redd.it/e0qmwmffs19h1.png?width=140&amp…
r/LocalLLaMA TIER_1 English(EN) · /u/rima_2711 · 2026-06-21 08:48

Gemma 4 QAT 在 KV 缓存量化方面似乎响应更好

<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1ubl0df/gemma_4_qat_seems_to_respond_significantly_better/"> <img alt="Gemma 4 QAT seems to respond significantly better to KV cache quantization" src="https://preview.redd.it/wxvhm0r1ml8h1.png?width=320&c…

报道来源 [8]

相关实体

相关话题