Researchers have identified a critical feature norm threshold, fn*, that largely dictates when neural collapse occurs in deep learning models. This threshold is specific to each model-dataset pair and is largely unaffected by training conditions, though training speed can vary. The study found that crossing this threshold consistently precedes neural collapse, acting as a practical predictor. Factors like network depth, activation functions, weight decay, and width all influence the speed of collapse and the value of fn*. AI
IMPACT Provides a new diagnostic tool for understanding and predicting representational reorganization in deep networks.
RANK_REASON This is a research paper published on arXiv detailing new findings about neural network dynamics. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →