English(EN)Beyond Retrieval: Progressive Latent Memory Evolution for Streaming Video Understanding
新AI模型通过高效内存管理增强流视频理解 · 跟踪7个来源
作者PulseAugur 编辑部·[8 个来源]·
研究人员正在开发用于流视频理解的新方法,重点关注多模态大语言模型(MLLMs)的高效内存管理。LatentStream和ShallowStream提出了超越简单存储和检索内存范式的新颖框架。LatentStream将历史证据内化为演化的潜在记忆,而ShallowStream使用浅层模型来高效编码帧和检索索引。其他方法,如CoFiE和StreamScout,也旨在通过解耦证据选择和自适应地决定何时更深入地处理帧来提高效率,同时在各种视频理解基准上取得最先进的结果。
AI
Streaming video understanding requires multimodal large language models (MLLMs) to process continuous visual inputs and respond to user queries under strict causality and bounded memory. Existing approaches typically compress historical observations into an external memory bank a…
arXiv cs.CL
TIER_1English(EN)·Jitai Hao, Ke Yang, Qiang Huang, Jun Yu·
arXiv:2609.02780v1 Announce Type: cross Abstract: Streaming video understanding is a critical capability for real-world applications, including embodied intelligence, autonomous driving, industrial monitoring, surveillance and early warning, and wearable assistants. However, proc…
LatentStream introduces a progressive latent working memory framework that internalizes streaming visual evidence into compact evolving tokens for continuous reasoning.
arXiv:2609.03675v1 Announce Type: new Abstract: Streaming video understanding requires Vision Language Models (VLLMs) to process growing video streams and answer user questions under tight latency constraints. Existing methods improve efficiency through token pruning and memory-b…
arXiv:2609.04131v1 Announce Type: new Abstract: Streaming video understanding requires multimodal large language models (MLLMs) to process continuous visual inputs and respond to user queries under strict causality and bounded memory. Existing approaches typically compress histor…
arXiv:2609.00291v1 Announce Type: new Abstract: Streaming video understanding requires answering questions that arrive at arbitrary moments over an unbounded video stream. Existing systems primarily focus on what to retain in a bounded memory, yet access that memory using the sam…
arXiv:2608.30294v1 Announce Type: new Abstract: Streaming video understanding requires answering questions at arbitrary times over a continuously growing visual stream. The central challenge is to compactly remember long-range history while effectively retrieving question-relevan…
arXiv:2608.27881v1 Announce Type: new Abstract: Recently, many streaming video understanding methods have been proposed by constructing an external memory to store historical data for computational reduction. Most methods focus on optimizing the injection procedure of current dat…