Looped Transformers
PulseAugur coverage of Looped Transformers — every cluster mentioning Looped Transformers across labs, papers, and developer communities, ranked by signal.
6 day(s) with sentiment data
-
LayerNorm acts as implicit gain control in looped transformers, research finds
A new research paper proposes that Layer Normalization in pre-LayerNorm looped transformers functions as an implicit gain controller. This mechanism helps stabilize the recurrence by coupling the block's local Lipschitz…
-
DiscoLoop architecture enhances multi-hop reasoning in LLMs
Researchers have developed DiscoLoop, a novel looping architecture designed to enhance multi-hop reasoning in large language models. Standard Transformers struggle with retaining information across multiple reasoning st…
-
Looped Transformers Stabilized with Learned Stochastic Stopping
Researchers have developed a method to stabilize extrapolation in Looped Transformers, a type of neural network architecture designed for variable-length algorithmic tasks. While these models can generalize well to long…
-
New research explores latent reasoning for LLMs, offering efficiency and interpretability gains
Two new research papers explore alternative methods for improving reasoning in large language models. One paper introduces LoTUS (Looped Transformers with parallel supervision on latents), a method using recurrent-depth…
-
New research explores merging large transformers and improving looped model stability
Two new research papers explore novel techniques for enhancing the capabilities and stability of large transformer models. The first paper introduces a scalable framework for linear mode connectivity (LMC) that allows f…
-
New LLM techniques enhance reasoning via iterative refinement and optimized looping · 5 sources tracked
Researchers have developed new methods to improve the reasoning capabilities of large language models (LLMs) through test-time scaling. The REVES framework uses a two-stage iterative process to augment training data and…
-
New methods boost LLM reasoning efficiency with compressed CoT
Researchers have developed new methods to improve the efficiency of chain-of-thought (CoT) reasoning in large language models. HybridThinker introduces a training scheme that balances retaining detailed thought steps wi…
-
Looped Transformers with Layer Norm Provably Learn Power Method
Researchers have theoretically demonstrated how looped transformers with layer normalization can learn the power method for principal component prediction. The study proves that such models, when trained with gradient d…
-
New dual-path architecture scales LLM compute and capacity
Researchers have introduced a novel dual-path architecture for large language models designed to efficiently scale both compute and capacity. This architecture utilizes a deep sublayer applied multiple times with shared…
-
Looped Transformers: A New Architecture for Enhanced Language Models
This article introduces the concept of looped transformers, a novel architecture for language models that aims to improve contextual understanding and dynamic representation. It explains how traditional transformer mode…