Neural collapse linked to class encoding in new research

Researchers have explored how label encoding influences neural collapse, a phenomenon observed in neural network classification models. Their study, using the unrestricted feature model with mean squared error training, found that for one-hot encoded labels and balanced data, class features transition from a simplex equiangular tight frame to an orthogonal frame with increased bias regularization. This structural change mirrors the orthogonal nature of one-hot encoded labels, suggesting a link between encoding and network behavior. AI

IMPACT Provides theoretical insights into neural network behavior, potentially informing future model design and training strategies.

RANK_REASON This is a research paper published on arXiv discussing a specific phenomenon in neural networks. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• class encoding
• mean squared error
• Neural collapse
• unrestricted feature model

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →