LightKV

LightKV reduces LVLM KV cache size and computation by compressing vision tokens

Researchers have developed LightKV, a new method to reduce the GPU memory overhead associated with Large Vision-Language Models (LVLMs). By exploiting redundancy in vision-token embeddings and using prompt-aware guidanc…