MomentumGD

Adam and Muon optimizers show implicit bias in neural networks

Researchers have analyzed the implicit bias of momentum-based optimizers like Adam and Muon when applied to smooth homogeneous neural networks. Their findings suggest that algorithms such as momentum steepest descent, i…