MuonBP

LionMuon optimizer cuts LLM pretraining costs with hybrid approach

Researchers have developed LionMuon, a novel optimizer designed to reduce the significant computational cost of pretraining large language models. LionMuon alternates between computationally expensive spectral steps, si…