Researchers have derived explicit layer-wise recursion relations for the tensors in the finite-width expansion of network statistics when using orthogonal initializations. This work provides a theoretical explanation for the stability of finite-width nonlinear networks initialized with orthogonal weights. The findings were validated experimentally through numerical solutions and analytical expansions that agreed well with Monte-Carlo estimates. AI
Summary written by gemini-2.5-flash-lite from 2 sources. How we write summaries →
IMPACT Provides a theoretical explanation for improved training performance in neural networks, potentially guiding future model architectures.
RANK_REASON This is a research paper published on arXiv detailing theoretical and experimental findings on neural network initialization.