Researchers have developed a new activation function called Heavy Tailed Activation Function (HTAF) to address the challenges of training neural networks with binary representations. HTAF is a smooth approximation of the Heaviside function, designed to maintain a large gradient mass for stable optimization. This new function enables the stable training of various neural network types, including Spiking Neural Networks and Binary Neural Networks, using gradient-based methods. The researchers also introduced Implicit Concept Bottleneck Models (ICBMs), which utilize HTAF to create interpretable image models with discrete feature representations, achieving performance comparable to or better than existing models. AI
影响 Enables more efficient and interpretable neural network training for specific applications.
排序理由 The cluster contains an academic paper detailing a new method for training neural networks.
- Heaviside function
- Heavy Tailed Activation Function (HTAF)
- Implicit Concept Bottleneck Models (ICBMs)
- Deep Heaviside neural Networks
- Spiking Neural Networks
- Binary Neural Networks
AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →
