eOptShrinkQ

HeadQ: Model-Visible Distortion and Score-Space Correction for KV-Cache Quantization

Researchers are developing several novel methods to optimize the Key-Value (KV) cache in large language models, which is a major bottleneck for long-context processing. These approaches include training models to inhere…