Weighted Reservoir Sampling from Distributed Streams

Nexus Sampling improves LLM KV cache eviction, reducing memory use

Researchers have developed Nexus Sampling, a novel method for managing KV cache eviction in large language models, particularly for long-context and agentic workloads. This training-free approach pairs Nexus scoring wit…