A new theoretical framework, the Orthogonal Polynomial Dynamics Framework, has been developed to explain why shallow neural networks prioritize learning low-frequency components, a phenomenon known as spectral bias. This framework analyzes the dynamics matrix H, which governs the gradient descent of Fourier series coefficients in an orthogonal polynomial basis. The research aims to provide a general explanation applicable to various activation functions and network architectures, moving beyond previous limitations in understanding spectral bias. AI
IMPACT Provides a theoretical basis for understanding and potentially controlling the learning process in neural networks.
RANK_REASON The item is an academic paper detailing a new theoretical framework for understanding neural network behavior. [lever_c_demoted from research: ic=1 ai=1.0]
- activation function
- arXiv
- dynamics matrix H
- eigenvalue
- eigenvector
- Fourier series
- F-Principle
- gradient descent
- Neural Networks
- Ordinary Differential Equations
- orthogonal polynomial basis
- rectifier
- sigmoid function
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
