Autoregressive Transformers
PulseAugur coverage of Autoregressive Transformers — every cluster mentioning Autoregressive Transformers across labs, papers, and developer communities, ranked by signal.
-
New metric quantifies 'cognitive fatigue' in language models
Researchers have introduced a new metric called the Fatigue Index (FI) to measure and diagnose "cognitive fatigue" in autoregressive language models. This phenomenon, characterized by degraded performance during long-ho…
-
Researchers analyze transformer expressivity in language modeling
Researchers have analyzed the probability distributions that autoregressive transformers can express, moving beyond their common treatment as language recognizers. The study reveals that making these models autoregressi…
-
Paper: Transformer Turing-completeness hinges on context management
A new paper argues that the Turing-completeness of autoregressive Transformers, commonly cited in AI research, is often misunderstood. The authors distinguish between fixed Transformer systems and scaling-family setting…