Lion
PulseAugur coverage of Lion — every cluster mentioning Lion across labs, papers, and developer communities, ranked by signal.
2 day(s) with sentiment data
-
LionMuon optimizer cuts training cost for large models
Researchers have introduced LionMuon, a novel optimization algorithm designed for efficient training of large-scale models. This method alternates between the low-cost updates of Lion and the stronger, albeit more expen…
-
Muown optimizer improves LLM training by controlling row-norm drift
Researchers have developed Muown, a novel optimization method designed to improve the training of large language models. Muown addresses issues with the Muon optimizer, specifically the upward drift of spectral norms in…
-
New LMO-IGT method accelerates optimization with implicit gradient transport
Researchers have introduced LMO-IGT, a novel class of stochastic optimization methods designed to accelerate convergence in machine learning. This approach leverages implicit gradient transport (IGT) to achieve faster r…
-
New Rose optimizer offers low VRAM, fast convergence, and great results
A new PyTorch optimizer named Rose has been released under the Apache 2.0 license. Developed by Matthew K., Rose is designed to be stateless, offering significantly lower VRAM usage compared to optimizers like AdamW, wi…