End-to-End Context Compression at Scale
Researchers have developed Latent Context Language Models (LCLMs), a new family of encoder-decoder compressors designed to address memory bottlenecks in long-context language model inference. Through extensive architecture search and pre-training on over 350 billion tokens, these models achieve compression ratios of 1:4, 1:8, and 1:16. LCLMs improve upon existing methods by enhancing general-task performance, compression speed, and reducing peak memory usage, making them efficient backbones for long-horizon agents. AI
IMPACT Introduces a new method for efficient long-context processing, potentially enabling more capable and less memory-intensive AI agents.