Longformer: The Long-Document Transformer
PulseAugur coverage of Longformer: The Long-Document Transformer — every cluster mentioning Longformer: The Long-Document Transformer across labs, papers, and developer communities, ranked by signal.
2 day(s) with sentiment data
-
Sliding-window attention slashes LLM costs, enabling longer context
Sliding-window attention is a technique that reduces the computational cost of large language models from quadratic to linear by restricting each token's attention to a local window of previous tokens. This method, empl…
-
Parameter-free sparse attention uses data compression for efficiency
Researchers have developed a novel parameter-free method for adaptive sparse attention in transformers, utilizing data compression techniques to dynamically select relevant content blocks for long-range attention. This …
-
LLMs tackle long-context challenge with attention tricks
The quadratic complexity of self-attention in large language models, where computation and memory scale with the square of the input token count, presents a significant challenge for processing long contexts. Various te…
-
New method uses reasoning graphs for robust LLM authorship attribution
Researchers have developed a novel method for attributing authorship of text generated by large language models (LLMs) by analyzing their reasoning structures. This approach utilizes reasoning graphs extracted via an ar…
-
Expanded dataset boosts transformer models in smishing detection
Researchers have developed COVA-X, an expanded dataset containing 10,985 synthetic conversations designed to detect multi-turn smishing attacks, particularly those targeting the elderly. This new dataset, an improvement…
-
New attention methods tackle LLM long-context challenges
Researchers are developing new attention mechanisms to handle increasingly long contexts in large language models. One approach, Runtime-Certified Bounded-Error Quantized Attention, uses tiered KV caches to compress mem…
-
CLaC system uses LLMs and encoders for political discourse clarity detection
Researchers presented a system for SemEval-2026 Task 6, focusing on detecting clarity and evasion in political discourse. Their approach involved comparing fine-tuned encoders with prompt-based large language models (LL…