Phase transition observed in neural network training dynamics

Researchers have identified a phase transition pattern in the approximation bounds and training errors of partially trainable neural networks. These networks combine fixed random features with adaptable representations, showing a rapid initial improvement in accuracy followed by diminishing returns. The study establishes a theoretical approximation rate for two-layer networks and verifies these assumptions with numerical experiments, demonstrating that a small proportion of trainable neurons can capture a significant portion of the accuracy gains. AI

IMPACT Provides theoretical insights into the training dynamics of neural networks, potentially informing future architectural designs.

RANK_REASON Academic paper on theoretical and numerical analysis of neural network properties. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• arXiv
• Hugging Face
• numerical analysis
• Phase Transition from Random Features to Fully Trainable Networks
• rectifier
• reproducing kernel Hilbert space

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →