Activation-Free Backbones for Image Recognition: Polynomial Alternatives within MetaFormer-Style Vision Models
Researchers have developed new activation-free backbone architectures for vision models, utilizing polynomial functions instead of traditional pointwise nonlinearities like ReLU or GELU. These novel modules, integrated into the MetaFormer framework, demonstrate competitive or superior performance compared to activation-based models on tasks such as ImageNet classification and semantic segmentation. The study also shows these polynomial variants outperform prior specialized polynomial networks while requiring less computational cost. AI
IMPACT Introduces a new architectural approach for vision models that could lead to more efficient and robust image recognition systems.