Boa Compressor

Boa Compressor offers linear-time context compression for LLMs

Researchers have developed the Boa Compressor, a novel method for compressing context in large language models (LLMs) that operates in linear time. Unlike previous methods that add latency, Boa uses a Mamba state-space …