UtVAA: Ultra-tiny Vision Transformer with Affix Attention for Mobile Image Classification
Researchers have developed UtVAA, an ultra-tiny Vision Transformer architecture optimized for mobile and edge devices. This new model incorporates Affix Attention, which combines local feature extraction with linear self-attention and coordinate attention for spatial modeling. UtVAA also utilizes Dilated Bottleneck blocks to expand receptive fields efficiently. The smallest variant boasts over 200,000 parameters and 53 million FLOPs, achieving competitive accuracy on benchmark datasets like CIFAR-10 and CIFAR-100, demonstrating that transformer-based vision models can be made significantly smaller without substantial performance loss. AI