Needle in a Haystack
PulseAugur coverage of Needle in a Haystack — every cluster mentioning Needle in a Haystack across labs, papers, and developer communities, ranked by signal.
-
Sliding Window Attention Outperforms Linear Attention in LLMs
A new research paper indicates that sliding window attention (SWA) with attention sinks performs as well as or better than linear attention models for large language models (LLMs). The study, published on arXiv and high…
-
New E^2-TTT method enhances long-context AI processing efficiency
Researchers have developed E^2-TTT, a novel method for Test-Time Training that balances expressivity and efficiency in long-context processing. This approach allows for parallelized chunk-level training while preserving…
-
SR-TTT Model Fails to Learn Retrieval, Paper Correction Reveals
A recent arXiv paper corrects previous findings regarding the SR-TTT model, demonstrating that it does not effectively learn retrieval mechanisms. The authors identify evaluation artifacts and a non-causal attention mec…
-
New benchmark PredicateLongBench probes LLM long-context limitations
Researchers have introduced PredicateLongBench, a new benchmark designed to evaluate the long-context capabilities of large language models by testing their ability to identify the longest contiguous subsequence of word…
-
SubQ unveils SubQ 1.1 Small with 12M-token context and sparse attention
SubQ has released its SubQ 1.1 Small model, featuring a new Subquadratic Sparse Attention (SSA) architecture designed to overcome the quadratic scaling limitations of traditional attention mechanisms. This new architect…
-
Subquadratic debuts 12M-token context window with linear scaling architecture
Subquadratic, a startup with 11 PhD researchers, has launched a new model featuring its Subquadratic Selective Attention (SSA) architecture, which claims to scale linearly with context length. This innovation allows for…