Spiking Transformers
PulseAugur coverage of Spiking Transformers — every cluster mentioning Spiking Transformers across labs, papers, and developer communities, ranked by signal.
- 2026-05-22 research_milestone A new framework was proposed to approximate nonlinear operators in Transformers for compatibility with spiking neural networks. source
-
New SAGE method improves Spiking Transformer training with adaptive gradients · 2 sources tracked
Researchers have introduced SAGE, a novel surrogate-gradient mechanism designed to improve the training of Spiking Transformers. This method leverages attention-guided entropy to adapt the surrogate-gradient slope durin…
-
New SCLA-BCP method enhances Spiking Transformer attention locality
Researchers have developed a new method called Spatially Contiguous Local Attention with Boundary Continuity Pathway (SCLA-BCP) to improve the spatial locality of Spiking Transformers. This approach addresses the challe…
-
New M-TTFS encoding boosts SNN energy efficiency for LLMs
Researchers have developed a new encoding method called Masked Time-to-First-Spike (M-TTFS) for spiking neural networks (SNNs) to improve energy efficiency in large language models. The M-TTFS encoding reassigns the sil…
-
New framework enables spiking neural networks for large language models
Researchers have developed a new framework to make large language models more compatible with neuromorphic hardware. The method focuses on creating spike-friendly approximations for the nonlinear operators within Transf…
-
Vision SmolMamba uses spike-guided pruning for energy-efficient vision models
Researchers have introduced Vision SmolMamba, a novel energy-efficient spiking state-space architecture designed for visual modeling. This architecture integrates spike-driven dynamics with linear-time selective recurre…