New mechanism explains one-third scaling in online classification

Researchers have identified a boundary-layer mechanism that explains a one-third scaling in online softmax classification. This mechanism shows that only examples near the teacher's decision boundaries contribute significantly to learning at later stages. The study predicts a power-law learning curve of \(\\alpha^{-1/3}\\) for test loss and generalization error, which is slower than the Bayes-optimal reference. They also suggest that learning-rate schedules can improve generalization error towards a \(\\alpha^{-1/2}\\) power law. AI

Summary written by gemini-2.5-flash-lite from 1 source. How we write summaries →

IMPACT Identifies a theoretical limitation in current classification methods and suggests potential improvements through learning-rate adjustments.

RANK_REASON Academic paper detailing a new theoretical mechanism for classification scaling. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• arXiv
• softma x classification

• paper
• other