wikitext
PulseAugur coverage of wikitext — every cluster mentioning wikitext across labs, papers, and developer communities, ranked by signal.
-
New theory predicts neural scaling laws from language statistics
Researchers have developed a new theory that can quantitatively predict the exponents of neural scaling laws for large language models trained on natural language datasets, particularly in data-limited scenarios. This t…
-
HOLA enhances linear attention models with a complementary memory system
Researchers have developed a novel approach called HOLA (Hippocampal Linear Attention) to enhance the memory capabilities of linear attention and state-space language models. This method introduces a complementary 'hipp…
-
New HOLA architecture enhances linear attention language models with dual memory system
Researchers have developed HOLA (Hippocampal Linear Attention), a novel architecture that enhances linear attention language models by incorporating a complementary memory system. This system addresses the issue of info…
-
New ROCKET-ActCost method shows trade-offs in LLM compression
Researchers have explored a new method for compressing large language models (LLMs) called ROCKET-ActCost, which aligns the allocation cost with an output-space objective. When applied to Qwen3-8B at 50% compression, RO…
-
New framework enables linear merging of billion-parameter transformers
Researchers have developed a new framework for merging large pretrained transformers, specifically those with billions of parameters. This method addresses limitations of previous approaches by optimizing interpolation …
-
New research explores merging large transformers and improving looped model stability
Two new research papers explore novel techniques for enhancing the capabilities and stability of large transformer models. The first paper introduces a scalable framework for linear mode connectivity (LMC) that allows f…
-
New methods like SMF and SAM reduce catastrophic forgetting in LLMs
Two new research papers explore methods to mitigate catastrophic forgetting in language models during fine-tuning. One paper introduces Sparse Memory Finetuning (SMF), which adds memory layers and updates only heavily a…