PulseAugur
EN
LIVE 05:20:06

New Math Model for Neural Network Initialization Spectra

By PulseAugur Editorial · [2 sources] ·

Researchers have developed a new mathematical framework to analyze the singular value spectrum of products of non-square random matrices. This framework is applicable to understanding the feature covariance eigenvalues of deep linear neural networks at initialization. The study introduces a geometric Dyson Brownian motion and a Burgers equation to model these spectral processes, ultimately yielding the free log-normal law. AI

IMPACT Provides a new theoretical lens for understanding the initialization dynamics of deep learning models.

RANK_REASON The cluster contains a research paper published on arXiv detailing new mathematical models for random matrices and neural networks.

Read on arXiv stat.ML →

AI-generated summary · Google Gemini · from 2 sources. How we write summaries →

New Math Model for Neural Network Initialization Spectra

COVERAGE [2]

  1. arXiv stat.ML TIER_1 English(EN) · Mufan Li, Jaume de Dios Pont, Mihai Nica, Daniel M. Roy ·

    Geometric Dyson Brownian Motions and the Free Log-Normal Limit for a Non-Square Product of Random Matrices

    arXiv:2606.30831v1 Announce Type: cross Abstract: We study the squared singular value spectrum of a product of non-square random matrices, a setting that also corresponds to the feature covariance eigenvalues of a deep linear neural network at initialization. We first take a prop…

  2. arXiv stat.ML TIER_1 English(EN) · Daniel M. Roy ·

    Geometric Dyson Brownian Motions and the Free Log-Normal Limit for a Non-Square Product of Random Matrices

    We study the squared singular value spectrum of a product of non-square random matrices, a setting that also corresponds to the feature covariance eigenvalues of a deep linear neural network at initialization. We first take a proportional depth-width $d,n$ limit with the number o…

RELATED ENTITIES

RELATED TOPICS