LongBench: a bilingual, multitask benchmark for long context understanding
PulseAugur coverage of LongBench: a bilingual, multitask benchmark for long context understanding — every cluster mentioning LongBench: a bilingual, multitask benchmark for long context understanding across labs, papers, and developer communities, ranked by signal.
7 day(s) with sentiment data
-
MoEZip: New KV Cache Compression for Sparse MoE LLMs
Researchers have developed MoEZip, a novel KV cache compression technique specifically designed for sparse Mixture-of-Experts (MoE) language models. Unlike existing methods that can degrade MoE performance by altering e…
-
Stratified Sparse Attention boosts LLM long-context inference speed and accuracy
Researchers have developed Stratified Sparse Attention (SSA), a new framework designed to improve the efficiency and accuracy of large language model inference over long contexts. SSA combines deterministic retrieval of…
-
Latent Compaction technique compresses LLM KV caches using test-time training
Researchers have developed a new technique called Latent Compaction, which uses test-time training to compress key-value (KV) caches in large language models. This method learns a document-specific compact cache by opti…
-
New research optimizes sparse attention for LLMs with QUILT and PageWeaver
Two new research papers, QUILT and PageWeaver, propose novel methods to optimize sparse attention mechanisms in large language models. QUILT introduces a workload-aware execution strategy that reuses shared KV entries a…
-
Attic-KV method improves LLM KV cache efficiency through self-quizzing
Researchers have developed a new method called Attic-KV that improves the efficiency of key-value (KV) caches in large language models. Unlike traditional methods that involve rereading the entire context, Attic-KV uses…
-
WakeKV introduces reactive KV cache residency for LLMs
Researchers have developed WakeKV, a novel KV-cache residency policy designed to improve the efficiency of large language models. Unlike existing methods that fix head classifications, WakeKV dynamically moves heads tha…
-
New HHR method boosts LLM long-context generation speed
Researchers have developed Hierarchical Hash Retrieval (HHR), a novel framework designed to enhance the efficiency of large language models (LLMs) during generation, particularly for long contexts. HHR addresses the lim…
-
New VLM frameworks LensVLM and FocusVTC improve visual text compression
Researchers have developed two new frameworks, LensVLM and FocusVTC, to improve how Vision Language Models (VLMs) handle long documents by compressing visual text representations. LensVLM uses a post-training recipe to …
-
New methods tackle LLM KV cache compression for efficient long-context inference · 10 sources tracked
Multiple research papers introduce novel techniques to compress the KV cache in large language models, a critical component for efficient inference with long contexts. These methods aim to reduce memory usage and improv…
-
New Self-Indexing Attention boosts LLM long-context inference speed
Researchers have developed a new framework called Self-Indexing Attention designed to improve the efficiency of sparse long-context Large Language Model (LLM) inference. This training-free method utilizes a shared trans…
-
KV Cache Compression Research Identifies Temporal Aggregation as Key Factor
Researchers have investigated the impact of temporal aggregation and ranking preservation on decoding-time KV cache compression in large language models. They found that exponential moving average (EMA) aggregation can …
-
New Faster Flash Decoding framework boosts LLM long-context efficiency
Researchers have developed Faster Flash Decoding (FFD), a new framework that significantly improves the efficiency of long-context decoding in Large Language Models. FFD addresses the memory bandwidth bottleneck and qua…
-
Tail-Replay boosts hybrid LLM inference speed by enabling unconstrained prefix reuse
Researchers have introduced Tail-Replay, a novel prefix caching mechanism designed to enhance the efficiency of hybrid large language models. These models combine full-attention and linear-attention layers to manage lon…
-
CRISP method boosts LLM inference speed and accuracy for long contexts
Researchers have developed CRISP, a novel method to improve the efficiency of long-context Large Language Model (LLM) inference. CRISP addresses the quadratic scaling bottleneck of self-attention during the prefilling p…
-
SEER framework enhances LLM long-context reasoning with selective visual-text compression
Researchers have developed SEER, a novel framework designed to improve long-context reasoning in large language models by selectively compressing visual-textual information. Unlike previous methods that applied uniform …
-
New AI architectures MARCH and Consolidator boost long-context memory
Two new research papers introduce novel architectures for enhancing long-context memory in AI models. The first, MARCH, proposes a system that periodically caches recurrent state checkpoints as "state anchors" to mainta…
-
New benchmarks test LLM long-context reasoning beyond simple retrieval
New benchmarks are emerging to test the capabilities of large language models (LLMs) in handling extended contexts, moving beyond simple "needle in a haystack" retrieval tests. While the needle test, popularized by Greg…
-
New ReTopK method accelerates long-context attention for LLMs
Researchers have developed ReTopK, a novel method to enhance the efficiency of long-context attention in transformer models. This technique reuses past attention decisions to speed up the process of identifying relevant…
-
PIVOT indexing method accelerates sparse attention in LLMs
Researchers have developed PIVOT, a novel indexing method designed to optimize token-level sparse attention in large language models. PIVOT addresses the bottleneck created by indexers in systems like DeepSeek Sparse At…
-
OpenAI subreddit user seeks chat evaluation benchmarks
A user on the r/OpenAI subreddit is seeking recommendations for datasets and benchmarks to evaluate chat model performance. They are specifically interested in measuring multi-turn accuracy and memory management, noting…