Grokking in ML requires breaking data symmetry for generalization

Researchers have investigated the phenomenon of grokking in machine learning, where a model achieves high training accuracy but only generalizes to new data much later. Their study, using the Recursive Feature Machine (RFM) algorithm on algebraic tasks, found that generalization is contingent upon breaking specific symmetries within the training dataset. The RFM algorithm appears to achieve this by recovering underlying invariance group actions inherent in the data, with learned feature matrices encoding elements of this invariance group, thus explaining the link between symmetry and generalization. AI

IMPACT Understanding grokking could lead to more robust and generalizable AI models by identifying key data properties.

RANK_REASON The cluster contains an academic paper detailing a new finding in machine learning. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv stat.ML →

• Grokking
• Marcel Tomas Bernal
• Power et al.
• Radhakrishnan et al.
• Recursive Feature Machine

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →