NorMuon
PulseAugur coverage of NorMuon — every cluster mentioning NorMuon across labs, papers, and developer communities, ranked by signal.
-
AF-Muon optimizer eliminates AdamW for tied-embedding models, saving memory
Researchers have developed AF-Muon, an optimizer that removes the need for AdamW in tied-embedding models, potentially saving around 20% of optimizer-state memory. This new approach maintains Muon's spectral-norm steepe…
-
Tilde Research launches Aurora optimizer to fix neuron death in Muon
Tilde Research has introduced Aurora, a novel optimizer designed to train neural networks more effectively. Aurora addresses a critical issue in the popular Muon optimizer where a significant number of neurons become pe…
-
Aurora optimizer boosts neural network training efficiency
Researchers have introduced Aurora, a new optimizer designed to improve the training of large neural networks, particularly those with rectangular matrices. Aurora addresses issues like neuron death in MLP layers that c…