DeiT-S
PulseAugur coverage of DeiT-S — every cluster mentioning DeiT-S across labs, papers, and developer communities, ranked by signal.
1 day(s) with sentiment data
-
Architecture search methods break down at high compression, researchers find
Researchers have identified a critical limitation in blockwise distillation methods used for architecture search, particularly in models like DeiT-S and DeiT-B. These methods, which score individual replacement blocks a…
-
New ViT token reduction schedule boosts out-of-distribution accuracy
Researchers have developed a new token reduction schedule for Vision Transformers (ViTs) that improves performance under distribution shift. This "late-concentrated" schedule, which removes more tokens in later layers, …
-
Foveated Dynamic Transformer mimics human vision for efficiency and robustness
Researchers have developed a Foveated Dynamic Transformer (FDT) inspired by the human visual system's foveated sampling and eye movements. This architecture dynamically selects tokens to process, enhancing efficiency an…
-
New depth pruning method boosts Vision Transformer efficiency
Researchers have developed a new method called HetDPT to improve depth pruning for Vision Transformers (ViTs). This approach accounts for the heterogeneity between different layers, which was a limitation in previous de…
-
Fusion framework unifies Vision Transformer adaptation for efficiency
Researchers have developed Fusion, a novel framework designed to enhance the efficiency of Vision Transformers (ViTs) by unifying sequential token adaptation techniques. This framework coordinates token merging, early e…
-
Vision Transformers reduce demographic bias in face anti-spoofing systems
A new study published on arXiv investigates the impact of Vision Transformer (ViT) architectures on demographic bias in face presentation attack detection (PAD) systems. The research compares ViTs against convolutional …