Withdrawn paper links Vision Transformer sparsity to data complexity

A recently withdrawn arXiv paper explored the phenomenon of "representational sparsity" in Vision Transformers (ViTs). The research, led by Kanishk Awadhiya, proposed that the observed "U-shaped" entropy profile in ViTs, where information is compressed in middle layers, is not an architectural flaw but a data-dependent adaptation. The study analyzed the Effective Encoding Dimension (EED) of DINO-trained ViTs across datasets of varying complexity, finding that the depth of this bottleneck correlates with the semantic abstraction required by the task. AI

IMPACT This research explored how Vision Transformers adapt their internal representations based on data complexity, potentially influencing future model design.

RANK_REASON The cluster contains a withdrawn academic paper discussing a technical aspect of Vision Transformers. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CV →

• arXiv
• CIFAR-100
• convolutional neural network
• Dino
• Kanishk Awadhiya
• Tiny-ImageNet
• University of California, Merced
• Vision Transformers

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →