New training method eliminates positional embeddings in Vision Transformers

Researchers have developed a new training technique called Active Spatial Guidance (Guidance) that eliminates the need for explicit positional embeddings in Vision Transformers (ViTs). By applying an auxiliary 2D coordinate-regression loss to the final-layer patch tokens during training, Guidance induces spatial organization directly from the data. This method consistently improved performance on tasks like ImageNet-100 classification and ADE20K semantic segmentation, outperforming traditional injected positional mechanisms such as learned absolute positional embeddings and rotary positional embeddings. AI

IMPACT This training technique could lead to more efficient and robust Vision Transformers by removing architectural complexity.

RANK_REASON The cluster contains an academic paper detailing a new method for training computer vision models. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CV →

• Active Spatial Guidance
• ADE20K
• DINOv3 ViT
• ImageNet-100
• learned absolute positional embeddings
• rotary positional embeddings
• Vision Transformers

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →