ReLU MLPs
PulseAugur coverage of ReLU MLPs — every cluster mentioning ReLU MLPs across labs, papers, and developer communities, ranked by signal.
-
Study reveals grokking in MLPs sharpens pre-existing algorithms
A new study published on arXiv investigates the phenomenon of 'grokking' in neural networks, specifically focusing on ReLU MLPs trained on modular arithmetic. Researchers found that instead of sinusoidal weight distribu…
-
Neural network architectures affect transfer specificity in implicit representations
Researchers have investigated how different neural network architectures impact the specificity of knowledge transfer in implicit neural representations. Their study compared SIREN, ReLU MLPs, and Fourier-feature MLPs a…
-
Grokking explained by two training clocks theory
Researchers have developed a theoretical framework to explain the phenomenon of "grokking" in machine learning, where a model fits training data and learns a generalizable rule at different rates. They propose the conce…