Transformer models show stratified residual streams anchored by prediction direction

Researchers have identified a phenomenon in trained transformer models where specific coordinate axes, termed privileged bases, exhibit distinct statistics compared to the rest of the residual stream. Analysis reveals that the prediction direction, which corresponds to the unembedding direction of the token the model is currently predicting, acts as a content-defined anchor. This anchor helps stratify the residual stream's variation based on its proximity to the prediction, with regions closer to the prediction being highly structured and those further away being flatter and less organized. AI

IMPACT Provides a deeper understanding of how transformer models process information and organize internal representations.

RANK_REASON Academic paper detailing a novel finding about transformer model internals. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• alphaXiv
• arXiv
• CatalyzeX
• CORE Recommender
• cs.CL
• DagsHub
• Gotit.pub
• Hugging Face
• machine learning
• ScienceCast
• Transformer

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →