SaluNet replaces normalization layers with learnable activation

Researchers have developed SaluNet, a novel deep network architecture that eliminates the need for traditional normalization layers like BatchNorm and LayerNorm. This is achieved through a new learnable activation function called SALU, which intrinsically stabilizes signals without relying on batch statistics. SaluNet demonstrates strong performance on image classification tasks, including CIFAR-10, CIFAR-100, and ImageNet, even at very small batch sizes where normalized networks typically fail. AI

IMPACT Enables more stable and adaptable deep network training, potentially improving performance in scenarios with limited batch sizes.

RANK_REASON The cluster contains a research paper introducing a novel deep learning architecture and activation function. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CV →

• CIFAR-10
• CIFAR-100
• ImageNet-1K
• LayerNorm
• SALU
• SaluNet

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →