Yiding Song

Model capacity's role in grokking explained by competing memorization and generalization speeds

A new research paper proposes an information-theoretic framework to explain how model capacity influences the phenomenon of "grokking" in machine learning. The study suggests that grokking, where a model suddenly genera…