Researchers have developed a Landau theory to explain the critical phenomenon observed in linear in-context learning (ICL). This theory frames the double-descent singularity, which occurs when pretraining samples approach the number of learnable parameters, as a critical phenomenon in quenched disordered systems. The study identifies sample-to-sample fluctuations of learned parameters as the root cause of this singularity and establishes a relationship between the order parameter and the empirical relaxation matrix eigenvalues. AI
IMPACT Provides a statistical-physics framework for understanding interpolation criticality in linear in-context learning.
RANK_REASON The cluster contains a research paper detailing a new theoretical framework for understanding a phenomenon in machine learning. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →