KV state
PulseAugur coverage of KV state — every cluster mentioning KV state across labs, papers, and developer communities, ranked by signal.
-
SAKURA architecture enables private image editing on phones
Researchers have developed SAKURA, a cloud-edge architecture designed to efficiently serve attention key-value (KV) states for private image editing on mobile devices. This system addresses the challenge of storing larg…
-
Prompt caching, not model routing, drives LLM cost savings
The primary driver of cost savings in large language models (LLMs) is prompt caching, rather than automatic model routing. Prompt caching stores key-value (KV) state specific to each model, meaning switching models resu…
-
New research questions reliability of language model credit estimation methods
A new paper from arXiv explores the reliability of counterfactual token-credit estimation in language models. The research highlights that re-feeding the transcript prefix as a fresh prompt, a common method, can introdu…