研究人员提出了一个单神经元模加傅里叶对齐假说的反例。该研究展示了一种情况，其中一个ReLU神经元在初始激活后变得不活跃，并在傅里叶能量均匀分布在频率类别上的极限处冻结。这种现象在Gaussian初始化下以正概率发生，并且可以扩展到Clarke轨迹，这表明在训练单个神经元进行模加时，单频率对齐并非普遍结果。 AI
影响 挑战了关于神经网络训练动态的假设，可能影响未来的模型架构和优化技术。
排序理由 该集群包含一篇在arXiv上发表的学术论文，详细介绍了神经网络训练中一个概念的理论反例。
在 arXiv cs.NE (Neural & Evolutionary) 阅读 →
AI 生成摘要 · Google Gemini · 来自 3 个来源。 我们如何撰写摘要 →
完整方法见我们的编辑标准。
arXiv:2608.04451v1 Announce Type: cross Abstract: We give a negative solution to MAIS-O60. We first construct an example in which an initially active ReLU neuron becomes completely inactive in finite time and thereafter remains frozen at a limit whose Fourier energy is equally di…
We give a negative solution to MAIS-O60. We first construct an example in which an initially active ReLU neuron becomes completely inactive in finite time and thereafter remains frozen at a limit whose Fourier energy is equally distributed among all nonzero real frequency classes…
We give a negative solution to MAIS-O60. We first construct an example in which an initially active ReLU neuron becomes completely inactive in finite time and thereafter remains frozen at a limit whose Fourier energy is equally distributed among all nonzero real frequency classes…