KV caches
PulseAugur coverage of KV caches — every cluster mentioning KV caches across labs, papers, and developer communities, ranked by signal.
7 day(s) with sentiment data
-
New RAG frameworks enhance AI generation with latent-space enrichment and personalized reasoning · 4 sources tracked
Researchers have developed several novel approaches to enhance retrieval-augmented generation (RAG) systems. CacheGraft utilizes latent-space document enrichment by encoding passages into fused KV cache representations,…
-
LSTM-Style Transformers Reduce Parameters and Cache Needs
Researchers have developed an LSTM-style transformer that reuses weights and shares a key-value cache across its recurrent core, significantly reducing parameter count and cache requirements. This approach allows for de…
-
New BeNice! method enhances AI inference memory efficiency
Researchers have developed a novel method called BeNice! to address the memory limitations of looped AI architectures during inference. This technique enables KV-sharing, where the key-value caches from later processing…
-
New Qwen model, video LoRA, and SVU research released; Anthropic users report faster limits
A new, smaller version of the Qwen model, Qwen3.5-4B-3D-Coder, has been released on Hugging Face, offering instructions for integration with various tools like llama.cpp, Ollama, and Docker. Separately, LiconStudio has …
-
CacheLink enables zero-prefill context handoff between LLMs
Researchers have developed CacheLink, a novel method for efficient cross-model context handoff in large language models. This technique allows a receiving model to continue processing context directly from a sender mode…
-
Latent Compaction technique compresses LLM KV caches using test-time training
Researchers have developed a new technique called Latent Compaction, which uses test-time training to compress key-value (KV) caches in large language models. This method learns a document-specific compact cache by opti…
-
Attic-KV method improves LLM KV cache efficiency through self-quizzing
Researchers have developed a new method called Attic-KV that improves the efficiency of key-value (KV) caches in large language models. Unlike traditional methods that involve rereading the entire context, Attic-KV uses…
-
Astera's Leo X boosts AI inference with fabric-attached DRAM
Astera has introduced its Leo X product, designed to enhance AI inference racks. This new technology utilizes fabric-attached DRAM to provide AI accelerators with increased space for KV caches, which is crucial for hand…
-
6 Architectural Shifts to Optimize LLM Pipelines for Cost and Latency
The article proposes six architectural shifts to optimize large language model (LLM) pipelines by reducing token costs and latency. It advocates for implementing strict retrieval-augmented generation (RAG) with vector d…
-
BinaryPC offers training-free sparse attention for efficient LLM decoding
Researchers have developed BinaryPC, a novel sparse attention mechanism designed to improve the efficiency of long-context large language models. This training-free method uses binary principal components to create comp…
-
Llama-server bug discarded KV caches, fix restores fast state restores
A bug in llama-server caused it to discard restored KV caches, forcing a full re-prefill and significantly increasing processing time. The issue stemmed from the server's state saving mechanism, which serialized token d…
-
InfoFlow KV improves retrieval-augmented generation for long contexts
Researchers have developed InfoFlow KV, a novel method for improving retrieval-augmented generation (RAG) in large language models. This technique addresses the bottleneck of prefilling large retrieved contexts during i…
-
TurboQuant compresses AI vectors to 2-4 bits without accuracy loss
A new method called TurboQuant has been developed to compress AI vectors, such as those in KV caches and attention keys, to as few as 2-4 bits per number without sacrificing accuracy. This technique relies on the princi…