KV retrieval

New SAKI method optimizes KV cache indexing for LLMs

Researchers have developed SAKI, a novel training-free method for optimizing KV cache indexing in large language models. SAKI directly preserves attention scores, outperforming existing techniques like principal compone…