English(EN)BeaconKV: Key-Value Cache Compression Guided by Beacon Queries for Efficient Large Reasoning Model Inference
新研究致力于 LLM KV 缓存压缩，实现高效长上下文推理 · 跟踪 5 个来源
作者PulseAugur 编辑部·[15 个来源]·
arXiv 上发布的多个研究论文提出了压缩大型语言模型键值（KV）缓存的新颖方法，以解决长上下文推理期间的内存瓶颈。BeaconKV、SGD-KV、VestigeKV 和 Random Attention 等技术旨在通过智能选择或压缩 KV 缓存数据来减少内存使用并提高吞吐量。这些方法包括使用信标查询和摘要引导的诊断、随机驱逐以及利用残余信号，所有这些方法都力求在显著降低内存需求的同时保持准确性。
AI
arXiv:2609.04971v1 Announce Type: cross Abstract: Large Reasoning Models (LRMs) achieve superior problem-solving through extended Chain-of-Thought (CoT) generation, but the resulting key-value (KV) cache grows linearly with sequence length and creates severe memory bottlenecks, o…
arXiv:2609.04263v1 Announce Type: cross Abstract: Low-bit key--value (KV) caches reduce the memory required for autoregressive decoding, but the resulting quality loss depends on the model and quantizer. We keep the quantizer fixed and distill the floating-cache model's behavior …
arXiv:2609.03949v1 Announce Type: cross Abstract: The problem. A long-lived KV cache must be compressed before the queries that will read it exist; selection by observed attention (H2O, SnapKV) collapses there (0.00-0.33 needle retrieval on a NoPE MLA model), because a token's im…
arXiv:2609.03235v1 Announce Type: new Abstract: Large language models (LLMs) face severe memory bottlenecks in long-context inference due to the linearly growing size of key-value (KV) caches. Existing KV cache compression techniques typically rely on simple heuristics, overlooki…
arXiv:2609.03430v1 Announce Type: new Abstract: Large language models achieve superior performance on tasks that require extended reasoning, but long chains of thought make the KV cache a severe memory bottleneck. Existing KV cache compression methods share one paradigm: score ea…
BeaconKV improves memory efficiency for long reasoning traces by using compact beacon queries to predict which past key-value pairs will be revisited, reducing cache size without sacrificing accuracy.
Large language models (LLMs) face severe memory bottlenecks in long-context inference due to the linearly growing size of key-value (KV) caches. Existing KV cache compression techniques typically rely on simple heuristics, overlooking the distinct functional roles of different at…
Random eviction of reasoning tokens matches selective KV cache compression because reasoning traces are self-protecting through redundancy, making scoring unnecessary once prompts are preserved.
arXiv:2609.00891v1 Announce Type: new Abstract: Sharing context between LLMs in a multi-model system requires the receiving model to prefill the shared prefix because KV caches are model-specific. Recent closed-form cross-model KV transfer, hereafter Full-Head Mapping, avoids thi…
arXiv:2604.08426v5 Announce Type: replace-cross Abstract: With the growing demand for long-context LLMs across a wide range of applications, the key-value (KV) cache has become a critical bottleneck for both latency and memory usage. Recently, KV-cache offloading has emerged as a…
arXiv cs.CL
TIER_1English(EN)·Renato Geh, Alex Chen, Daniel Israel, Aditya Grover, Guy Van den Broeck·
arXiv:2608.28293v1 Announce Type: new Abstract: The premise and promise of KV (cache) eviction is simple: higher throughput can be achieved by evicting some entries from the KV cache, at a negligible cost to quality. This holds empirically for many existing methods, though most r…
<p>The latency and energy bottleneck in small-model inference often lies not in compute but in memory access and VRAM management. A well-designed KV Cache tiering and disaggregated storage architecture can improve both time-to-first-token (TTFT) and energy per unit of throughput.…
dev.to — LLM tag
TIER_1English(EN)·Mingxin Technology·
<p>Spatial locality optimization for KV Cache is evolving from in-HBM paging management to a tiered architecture spanning multiple storage layers. Measured on the Mingxin FX100 under a 480B production deployment with long-context cold-recovery workloads, tiered acceleration impro…
<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1w8jflp/block_kv_cache_streaming_bound_vram_at_long/"> <img alt="Block KV cache streaming: bound VRAM at long context via a shared CUDA phase arena by giveen · Pull Request #357 · TheTom/llama-cpp-turboquant" …
