本文研究了线性宽度的两层神经网络中的特征学习，比较了两次梯度下降与一次梯度下降的影响。研究提供了更新权重的详细谱表征，揭示它们形成一个具有多个学习方向的尖峰随机矩阵。研究强调，重用批次可以捕获超出单一信息指数的方向，这一优势延伸到了高维极限。 AI
影响 为理解过参数化网络中的优化和特征学习提供了数学框架。
排序理由 在arXiv上发表的学术论文，详细介绍了神经网络特征学习的理论进展。
AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →
线性宽度网络中两次梯度下降可增强特征学习
报道来源 [2]
-
Feature Learning in Linear-Width Two-Layer Networks: Two vs. One Step of Gradient Descent
arXiv:2605.17767v1 Announce Type: new Abstract: We study feature learning in two-layer neural networks within the linear-width regime, where the number of hidden neurons, sample size, and input dimension scale proportionally. While recent work has analyzed feature learning via a …
-
Feature Learning in Linear-Width Two-Layer Networks: Two vs. One Step of Gradient Descent
We study feature learning in two-layer neural networks within the linear-width regime, where the number of hidden neurons, sample size, and input dimension scale proportionally. While recent work has analyzed feature learning via a single step of gradient descent, such updates ar…