PulseAugur
中
实时 18:21:12
English(EN) Width and Depth Limits Commute During Training

arXiv论文：宽度和深度限制在训练过程中可交换

作者 PulseAugur 编辑部 · [1 个来源] ·

研究人员证明，残差网络的无限宽度和无限深度限制不仅在初始化时，而且在训练过程中也具有可交换性。这种可交换性对于具有特定缩放条件的线性残差网络成立，包括深度-"mu"P缩放和$\mu$P宽度参数化。该研究证明，对于具有梯度裁剪的固定数量梯度下降步数，当宽度和深度都趋于无穷大时，网络输出在L2范数上收敛，将可交换性结果扩展到任何有限的训练范围。 AI

影响 为理解大型神经网络在训练过程中的行为提供了理论见解，可能影响未来的模型架构。

排序理由 详细介绍神经网络训练动力学理论进展的学术论文。[lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.LG 阅读 →

AI 生成摘要 · Google Gemini · 来自 1 个来源。 我们如何撰写摘要 →

arXiv论文：宽度和深度限制在训练过程中可交换

本文如何被排名

Signal score
4 / 100
Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket
Tool
详细介绍神经网络训练动力学理论进展的学术论文。[lever_c_demoted from research: ic=1 ai=1.0]
Source corroboration
Single-source cluster
Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics
paper, other
Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance
High
Clearly on-topic for AI-industry coverage.
Story freshness
Same-day
Cluster formed today. Ranking reflects the current source set at time of score.

完整方法见我们的编辑标准。

报道来源 [1]

  1. arXiv cs.LG TIER_1 English(EN) · Ruijia Zhang, Soufiane Hayou ·

    Width and Depth Limits Commute During Training

    arXiv:2610.30729v1 Announce Type: new Abstract: Recent work established that the infinite-width and infinite-depth limits of residual networks commute at initialization when residual branches are scaled by $1/\sqrt{depth}$. In particular, the network output converges to the same …

相关实体

相关话题