Analysis of Metastable States in the Transformer Activation Space
Researchers have conducted experiments to analyze metastable states within the activation space of trained Transformer models. The study confirmed that tokens cluster into persistent groups across layers, mirroring predictions from a theoretical dynamical system model of attention. However, the experiments falsified key aspects of the theory, finding that the energy driving the clustering is not monotonic and that collapse speed is determined by the value matrix rather than model depth. AI
IMPACT Confirms theoretical predictions about token clustering in Transformers while identifying limitations in the underlying energy model.