Signum
PulseAugur coverage of Signum — every cluster mentioning Signum across labs, papers, and developer communities, ranked by signal.
-
LionMuon optimizer cuts LLM pretraining costs with hybrid approach
Researchers have developed LionMuon, a novel optimizer designed to reduce the significant computational cost of pretraining large language models. LionMuon alternates between computationally expensive spectral steps, si…
-
LionMuon optimizer offers efficient large-scale model training
Researchers have introduced LionMuon, a novel optimizer designed for efficient large-scale training of machine learning models. This new method alternates between Lion's and Muon's update steps, utilizing a shared momen…
-
New adaptive batch sizing method cuts training steps by up to 66%
Researchers have developed a new method for adaptive batch sizing in machine learning that accounts for the non-Euclidean geometry of optimizers like signSGD and spectral descent. This approach, which estimates non-Eucl…
-
Adam and Muon optimizers show implicit bias in neural networks
Researchers have analyzed the implicit bias of momentum-based optimizers like Adam and Muon when applied to smooth homogeneous neural networks. Their findings suggest that algorithms such as momentum steepest descent, i…
-
LionMuon optimizer cuts training cost for large models
Researchers have introduced LionMuon, a novel optimization algorithm designed for efficient training of large-scale models. This method alternates between the low-cost updates of Lion and the stronger, albeit more expen…