modular arithmetic
PulseAugur coverage of modular arithmetic — every cluster mentioning modular arithmetic across labs, papers, and developer communities, ranked by signal.
-
New 'magnitude potential' metric aids neural network generalization analysis
Researchers have introduced a new concept called "magnitude potential" derived from metric magnitude theory to better understand generalization in neural networks. This quantity, calculated at the logit layer, reflects …
-
New Theory Explains "Grokking" in Machine Learning Models
Researchers have developed a statistical theory to explain the phenomenon of "grokking," where a model learns underlying signals at a different stage than it fits training data. The theory characterizes how regularizati…
-
Grokking transition in neural networks quantified, data complexity key
Researchers have quantified the transition from memorization to generalization in neural networks, a phenomenon known as grokking. They discovered a power-law scaling relation for the onset time of grokking, indicating …
-
New model explains neural network grokking by focusing on representability
Researchers have developed a new model to understand grokking in neural networks, a phenomenon where generalization is delayed. This model, using holomorphic monomial activations on modular arithmetic tasks, demonstrate…
-
New framework explains system convergence via phase transitions
Researchers have proposed a new framework called the Hierarchical Emergence Framework (HEF) to explain how complex systems, from machine learning to biology, converge on similar high-level structures. HEF models emergen…
-
Weight decay controls transformer training regimes, new diagnostics revealed
Researchers have identified weight decay as a key parameter controlling the training regimes of transformers on modular arithmetic tasks. They introduced two new, low-cost online diagnostics—mean pairwise attention-head…