Researchers have developed MERIT, a novel framework for understanding ultra-long videos that exceed practical processing limits for current multi-modal large language models. MERIT employs a two-stage approach, first constructing a query-agnostic memory with a focus on retrievability and then performing retrieval-based inference. This method utilizes an episodic multi-key representation for precise memory retrieval and a neighbor filtering mechanism for capturing broader semantic context by expanding temporal scope around retrieved segments at inference time. MERIT has demonstrated state-of-the-art performance on benchmarks such as EgoLifeQA, LVBench, and Video-MME (Long). AI
IMPACT This framework could enable new applications in analyzing extended video content, such as surveillance footage or historical archives.
RANK_REASON The item is an academic paper detailing a new framework for video understanding. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
