New 'magnitude potential' metric aids neural network generalization analysis

Researchers have introduced a new concept called "magnitude potential" derived from metric magnitude theory to better understand generalization in neural networks. This quantity, calculated at the logit layer, reflects how well a point is represented within a dataset. Experiments show that ratios of this magnitude potential correlate with memorization scores and can detect structural changes in decision boundaries, serving as a geometric indicator of grokking in modular arithmetic tasks. The magnitude potential also remains informative about geometric structure even when neural collapse is suppressed. AI

IMPACT Introduces a novel metric for analyzing neural network generalization and training dynamics.

RANK_REASON The cluster contains a research paper detailing a new theoretical concept for analyzing neural network generalization. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• arXiv
• Decision boundaries and receiver operating characteristic curves: New methods for determining accelerometer cutpoints
• Feldman memorization scores
• Generalization in Neural Networks Through the Lens of Magnitude Potential
• grokking
• Lisa Feldman Barrett
• logit layer
• Magnitude Potential
• metric magnitude
• modular arithmetic
• neural collapse
• Neural Networks

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →