New theories explore spectral dynamics in deep neural network training

作者 PulseAugur 编辑部 · [5 个来源] · 2026-05-08 15:28

Two new arXiv papers explore the spectral dynamics of deep neural networks during training. One paper introduces "Neural Low-Degree Filtering" (Neural LoFi) as a theoretical framework to understand hierarchical feature learning as an iterative spectral procedure. The other paper uses a dynamical mean-field theory to analyze how hidden-weight spectra evolve, predicting outlier behavior and hyperparameter transfer in wide networks. AI

影响 These theoretical frameworks offer new perspectives on how deep neural networks learn, potentially guiding future model development and analysis.

排序理由 Two academic papers published on arXiv presenting theoretical frameworks for understanding deep learning.

在 arXiv cs.AI 阅读 →

AI 生成摘要 · Google Gemini · 来自 5 个来源。我们如何撰写摘要 →

报道来源 [5]

arXiv cs.LG TIER_1 English(EN) · Florent Krzakala · 2026-05-13 14:44

Deep Learning as Neural Low-Degree Filtering: A Spectral Theory of Hierarchical Feature Learning

Understanding how deep neural networks learn useful internal representations from data remains a central open problem in the theory of deep learning. We introduce Neural Low-Degree Filtering (Neural LoFi), a stylized limit of gradient-based training in which hierarchical feature …
Hugging Face Daily Papers TIER_1 English(EN) · 2026-05-13 14:44

Deep Learning as Neural Low-Degree Filtering: A Spectral Theory of Hierarchical Feature Learning

Understanding how deep neural networks learn useful internal representations from data remains a central open problem in the theory of deep learning. We introduce Neural Low-Degree Filtering (Neural LoFi), a stylized limit of gradient-based training in which hierarchical feature …
arXiv cs.AI TIER_1 English(EN) · Blake Bordelon · 2026-05-08 15:28

Spectral Dynamics in Deep Networks: Feature Learning, Outlier Escape, and Learning Rate Transfer

We study the evolution of hidden-weight spectra in wide neural networks trained by (stochastic) gradient descent. We develop a two-level dynamical mean-field theory (DMFT) that jointly tracks bulk and outlier spectral dynamics for spiked ensembles whose spike directions remain st…
arXiv stat.ML TIER_1 English(EN) · Yatin Dandi, Matteo Vilucchio, Luca Arnaboldi, Hugo Tabanelli, Florent Krzakala · 2026-05-14 04:00

Deep Learning as Neural Low-Degree Filtering: A Spectral Theory of Hierarchical Feature Learning

arXiv:2605.13612v1 Announce Type: cross Abstract: Understanding how deep neural networks learn useful internal representations from data remains a central open problem in the theory of deep learning. We introduce Neural Low-Degree Filtering (Neural LoFi), a stylized limit of grad…
arXiv stat.ML TIER_1 English(EN) · Clarissa Lauditi, Cengiz Pehlevan, Blake Bordelon · 2026-05-11 04:00

Spectral Dynamics in Deep Networks: Feature Learning, Outlier Escape, and Learning Rate Transfer

arXiv:2605.07870v1 Announce Type: cross Abstract: We study the evolution of hidden-weight spectra in wide neural networks trained by (stochastic) gradient descent. We develop a two-level dynamical mean-field theory (DMFT) that jointly tracks bulk and outlier spectral dynamics for…

报道来源 [5]

Deep Learning as Neural Low-Degree Filtering: A Spectral Theory of Hierarchical Feature Learning

Deep Learning as Neural Low-Degree Filtering: A Spectral Theory of Hierarchical Feature Learning

Spectral Dynamics in Deep Networks: Feature Learning, Outlier Escape, and Learning Rate Transfer

Deep Learning as Neural Low-Degree Filtering: A Spectral Theory of Hierarchical Feature Learning

Spectral Dynamics in Deep Networks: Feature Learning, Outlier Escape, and Learning Rate Transfer

相关实体

相关话题