RedKnot

New systems tackle LLM long-context serving bottlenecks

Two new research papers introduce novel approaches to managing the KV cache, a critical bottleneck in serving large language models with long contexts. RedKnot proposes a head-aware KV cache management system that decom…