PulseAugur
EN
LIVE 12:27:14

New model explains neural network grokking by focusing on representability

By PulseAugur Editorial · [3 sources] ·

Researchers have developed a new model to understand grokking in neural networks, a phenomenon where generalization is delayed. This model, using holomorphic monomial activations on modular arithmetic tasks, demonstrates that representability is key. When a network's expressible function class collapses to an algebraic variety, tasks are either instantly solved or impossible to fit, eliminating the typical grokking regime. The study shows a 99.8% accuracy in predicting these outcomes across 585 runs, offering a new perspective on the capacity-grokking relationship. AI

IMPACT Provides a theoretical framework for understanding generalization in neural networks, potentially informing future model design.

RANK_REASON The cluster contains an academic paper detailing a new model and theoretical findings in machine learning.

Read on Hugging Face Daily Papers →

AI-generated summary · Google Gemini · from 3 sources. How we write summaries →

New model explains neural network grokking by focusing on representability

COVERAGE [3]

  1. Hugging Face Daily Papers TIER_1 English(EN) ·

    Algebraic Representability as the Limiting Regime of Grokking: An Exactly Solvable Model with Holomorphic Activations

    Neural networks trained on modular arithmetic exhibit grokking, a delayed transition from memorisation to generalisation known to depend on model capacity: too little and the network memorises slowly or not at all, too much and it generalises almost immediately. What happens at t…

  2. arXiv stat.ML TIER_1 English(EN) · Chon-Fai Kam, Xavier Cadet, Miloud Bessafi, Frederic Cadet ·

    Algebraic Representability as the Limiting Regime of Grokking: An Exactly Solvable Model with Holomorphic Activations

    arXiv:2607.13749v1 Announce Type: cross Abstract: Neural networks trained on modular arithmetic exhibit grokking, a delayed transition from memorisation to generalisation known to depend on model capacity: too little and the network memorises slowly or not at all, too much and it…

  3. arXiv stat.ML TIER_1 English(EN) · Frederic Cadet ·

    Algebraic Representability as the Limiting Regime of Grokking: An Exactly Solvable Model with Holomorphic Activations

    Neural networks trained on modular arithmetic exhibit grokking, a delayed transition from memorisation to generalisation known to depend on model capacity: too little and the network memorises slowly or not at all, too much and it generalises almost immediately. What happens at t…

RELATED TOPICS