KV caching
PulseAugur coverage of KV caching — every cluster mentioning KV caching across labs, papers, and developer communities, ranked by signal.
4 day(s) with sentiment data
-
Ripple-Pivot Search accelerates Diffusion LLM inference by up to 18x
Researchers have introduced Ripple-Pivot Search (RPS), a new decoding method for Diffusion Large Language Models (dLLMs) that significantly speeds up inference. RPS exploits a "ripple effect" where committing to a mid-e…
-
Diffusion language models research tackles efficiency and confidence gaps · 6 sources tracked
Recent research explores methods to improve the efficiency and effectiveness of diffusion language models (DLMs). One paper investigates when classifier-free guidance (CFG) is truly necessary during decoding, suggesting…
-
Comprehension Memory slashes LLM context costs, boosting efficiency
A new paper introduces Comprehension Memory (CoMem), a technique designed to significantly reduce the memory and computational costs associated with long-context language models. CoMem operates by caching intermediate l…
-
New MM-ShiftKV method optimizes KV caching for multimodal LLMs
Researchers have developed MM-ShiftKV, a novel method for optimizing Key-Value (KV) caching in multimodal large language models (MLLMs). This technique addresses the issue where prefill-stage KV selection methods, which…
-
Apple researchers advance diffusion language models with new decoding techniques
Apple's Machine Learning Research division has published several papers detailing advancements in diffusion language models (dLLMs). These models offer potential for faster inference compared to autoregressive models by…
-
New Diffusion Models Enable Real-Time AI Music Generation on Consumer Hardware
Researchers have developed Live Music Diffusion Models (LMDMs), a novel approach to interactive music generation using diffusion models that can run on consumer hardware. LMDMs improve upon existing methods by optimizin…
-
LLM KV Caching Explained: Speed vs. Memory Tradeoff
Large language models utilize KV caching to accelerate inference by storing previously computed key and value vectors, rather than recomputing them for each new token. This technique significantly speeds up token genera…
-
Stochastic KV Routing enables adaptive depth-wise cache sharing for LLMs
Researchers have developed a new method called Stochastic KV Routing to reduce the memory footprint of transformer language models. This technique enables adaptive depth-wise cache sharing by training layers to randomly…