SketchSSM

SketchSSM method boosts LLM state-read efficiency by 10x

Researchers have introduced SketchSSM, a novel method that enhances the efficiency of hybrid-attention models by approximating state reads. This technique reduces KV-cache growth and enables larger decode batches by buf…