Chenwei Tang

New theory unifies KV cache eviction for LLMs, improving long-context generation

Researchers have developed a new method for managing KV cache eviction in large language models, drawing inspiration from the Information Bottleneck principle. This approach, named CapKV, aims to preserve the most predi…