研究人员开发了 SpikeDecoder，这是一种使用脉冲神经网络 (SNN) 实现的 Transformer 解码器块的新方法，用于自然语言处理。该方法旨在显著降低传统 Transformer 模型的高能耗。实验表明，与人工神经网络 (ANN) 相比，SpikeDecoder 可将理论能耗降低 87% 至 93%，同时还探索了各种嵌入方法和架构修改。 AI
影响 脉冲神经网络为大幅降低大型语言模型的能耗提供了一条途径。
排序理由 该集群包含一篇详细介绍新模型架构的学术论文。
在 arXiv cs.NE (Neural & Evolutionary) 阅读 →
AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →
arXiv:2606.12287v1 Announce Type: cross Abstract: The Transformer architecture is widely regarded as the most powerful tool for natural language processing, but due to a high number of complex operations, it inherently faces the issue of high energy consumption. To address this i…
The Transformer architecture is widely regarded as the most powerful tool for natural language processing, but due to a high number of complex operations, it inherently faces the issue of high energy consumption. To address this issue, we consider Spiking Neural Networks (SNNs), …