ViT depth computation approximated by linear dynamics

Researchers have explored the internal computations of Vision Transformers (ViTs) by applying Dynamic Mode Decomposition (DMD). Their findings suggest that contiguous blocks within a ViT can be approximated by a single linear operator applied repeatedly. This linear operator accurately predicts intermediate activations over short spans, particularly in earlier layers and for the 'cls' token, but this local fidelity does not translate to improved performance on downstream tasks. AI

Summary written by gemini-2.5-flash-lite from 1 source. How we write summaries →

IMPACT Reveals that ViT computations exhibit linear dynamics, suggesting potential for model compression and efficiency gains.

RANK_REASON The cluster contains an academic paper detailing a new analytical method for understanding existing models. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CV →

• Vision Transformers
• Dynamic Mode Decomposition
• DINOv3-H/16+
• Nishant Suresh Aswani

• paper
• model release