New Crinit initialization rule improves deep neural network training

Researchers have developed a new initialization rule called Crinit for deep neural networks, which precisely places them on their finite-width edge of chaos. This rule is derived from analyzing the distance between inputs as a random walk, accounting for the shift in the edge of chaos at finite widths. Simulations and experiments on MNIST and Fashion-MNIST datasets show that Crinit outperforms standard initialization rules, particularly in deeper networks where other methods often fail, leading to more accurate training and fewer steps. AI

IMPACT Offers a more robust initialization strategy for deep learning models, potentially improving performance and training efficiency.

RANK_REASON Academic paper detailing a new method for deep neural networks. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• arXiv
• deep neural networks
• Fashion-MNIST
• MNIST database

• paper
• infra

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →