Researchers have demonstrated that delayed generalization, a phenomenon where training accuracy significantly outpaces test accuracy, can occur even in simplified linear models. This effect arises when using gradient descent with an exponential-tail loss on linearly separable data. The study characterizes the process into three phases: population, support vector-dominated unlearning, and generalization, with the convergence of the residual beyond the maximum-margin direction initiating the unlearning phase. These dynamics were validated through experiments and observed in a deep model using the MNIST dataset. AI
IMPACT Provides theoretical insights into model generalization dynamics, potentially informing future model design and training strategies.
RANK_REASON Academic paper detailing theoretical findings and experimental validation. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
