Vision Transformers learn spatial hierarchy mirroring primate visual cortex

Researchers have investigated how Vision Transformers (ViTs) encode spatial information without explicit spatial supervision during pretraining. By probing a ViT-B/16 model, they found that boundary structure is decodable by layer 5-6, while depth information, requiring more global cues, becomes decodable two to three layers later. This learned spatial hierarchy within the ViT mirrors the progression observed in the primate visual cortex. AI

IMPACT Reveals how classification-trained ViTs develop an internal spatial hierarchy, potentially informing future model architectures.

RANK_REASON Academic paper analyzing the internal workings of Vision Transformers.

Read on arXiv cs.CV →

• BSDS500
• NYU Depth V2
• ViT-B/16
• Vision Transformers

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →