Visual KV-cache retention is task-inert, new paper finds

A new research paper challenges the assumption that visual key-value (KV) caches in vision-language models retain task-relevant information. The study found that the amount of visual content retained in the KV cache is largely task-inert and does not correlate with whether the information is causally used for answering questions. While attention mechanisms showed a weak correlation with causal utilization, pixel-decodable reconstructability proved to be a poor signal for KV cache compression, with one model retaining 2.7 times more task-inert content than another. AI

IMPACT Challenges current assumptions about KV cache efficiency and suggests new directions for optimizing vision-language model performance.

RANK_REASON Research paper published on arXiv detailing findings about visual KV-cache in vision-language models. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CV →

• attention
• encoder-based model
• encoder-free model
• KV ablation
• KV cache
• Pixel Decodability Is Not a Compression Signal: Causally Evaluating Importance Proxies for Visual KV-Cache Eviction
• pixel-inversion decoder
• super-patch

• paper
• infra

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →