SGDW

Muon optimizer shows promise in theoretical and practical neural network training

Two new research papers explore the Muon optimizer, an approach designed to better handle matrix-structured parameters in neural networks. The first paper introduces a matrix-aware geometry for Sharpness-Aware Minimizat…