研究人员推出了一种新颖的神经网络层ManifoldFlow，它比传统的Stiefel层提供了更大的灵活性。虽然Stiefel层强制执行固定的奇异值，但ManifoldFlow允许使用可学习的正谱，从而能够实现奇异值的方向相关衰减或放大。这种方法在各种实验中显示出改进，特别是在循环语言模型投影中，表明它在需要正交基但固定谱过于受限的情况下很有用。 AI
影响 为神经网络权重引入了更灵活的谱控制机制，有望提高语言模型和其他基于序列的任务的性能。
排序理由 该集群包含一篇详细介绍神经网络新层的研究论文。[lever_c_demoted from research: ic=1 ai=1.0]
AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →
arXiv:2607.04535v1 Announce Type: cross Abstract: Orthogonal and Stiefel layers give neural weights exact spectral control, but they also impose a strong modeling constraint: all represented singular values are fixed at one. Many settings that benefit from an orthonormal basis st…
Orthogonal and Stiefel layers give neural weights exact spectral control, but they also impose a strong modeling constraint: all represented singular values are fixed at one. Many settings that benefit from an orthonormal basis still need direction-dependent attenuation or amplif…