KV caches
PulseAugur coverage of KV caches — every cluster mentioning KV caches across labs, papers, and developer communities, ranked by signal.
1 day(s) with sentiment data
-
BinaryPC offers training-free sparse attention for efficient LLM decoding
Researchers have developed BinaryPC, a novel sparse attention mechanism designed to improve the efficiency of long-context large language models. This training-free method uses binary principal components to create comp…
-
Llama-server bug discarded KV caches, fix restores fast state restores
A bug in llama-server caused it to discard restored KV caches, forcing a full re-prefill and significantly increasing processing time. The issue stemmed from the server's state saving mechanism, which serialized token d…
-
InfoFlow KV improves retrieval-augmented generation for long contexts
Researchers have developed InfoFlow KV, a novel method for improving retrieval-augmented generation (RAG) in large language models. This technique addresses the bottleneck of prefilling large retrieved contexts during i…
-
TurboQuant compresses AI vectors to 2-4 bits without accuracy loss
A new method called TurboQuant has been developed to compress AI vectors, such as those in KV caches and attention keys, to as few as 2-4 bits per number without sacrificing accuracy. This technique relies on the princi…