Autoregressive Transformers
PulseAugur coverage of Autoregressive Transformers — every cluster mentioning Autoregressive Transformers across labs, papers, and developer communities, ranked by signal.
-
AI models to speed up collider event generation for LHC
Researchers have developed a new method for generating collider event data using autoregressive transformers and residual-quantized tokens. This approach aims to address the computational bottlenecks expected at the Hig…
-
Autoregressive transformers show surprising ability to predict chaotic dynamics
A new research paper explores the ability of autoregressive transformers to understand and predict complex chaotic systems, even when trained on limited data. The study demonstrates that these models, when exposed to re…
-
New research tackles evaluation and architecture for masked diffusion language models
Two new research papers introduce novel evaluation protocols and architectures for masked diffusion language models (MDLMs). The first paper, "CaRE," proposes a compute-aware framework to standardize evaluations, reveal…
-
New metric quantifies 'cognitive fatigue' in language models
Researchers have introduced a new metric called the Fatigue Index (FI) to measure and diagnose "cognitive fatigue" in autoregressive language models. This phenomenon, characterized by degraded performance during long-ho…
-
Researchers analyze transformer expressivity in language modeling
Researchers have analyzed the probability distributions that autoregressive transformers can express, moving beyond their common treatment as language recognizers. The study reveals that making these models autoregressi…
-
Paper: Transformer Turing-completeness hinges on context management
A new paper argues that the Turing-completeness of autoregressive Transformers, commonly cited in AI research, is often misunderstood. The authors distinguish between fixed Transformer systems and scaling-family setting…