arXiv:2610.07103v1 Announce Type: cross Abstract: Muon, an optimizer known for its efficiency, has a clear interpretation for matrix-valued updates, but convolutional kernels are stored as four-dimensional tensors. Standard implementations reshape these tensors into matrices, a s…
arXiv:2610.03306v1 Announce Type: cross Abstract: Existing convergence analyses of Muon either assume exact orthogonalization or analyze classical Newton--Schulz polynomials, and guarantee only stationarity, so it is unresolved what Muon's five tuned Newton--Schulz steps preserve…
arXiv:2610.02798v1 Announce Type: new Abstract: The Muon optimizer applies spectral orthogonalization to matrix-valued updates and has shown strong performance in large-scale neural network training, yet the mechanisms of this transformation in feature learning remain poorly unde…
arXiv:2610.01395v1 Announce Type: new Abstract: Muon improves large-scale training by applying a spectral-norm steepest-descent update to matrix parameters, but practical models also contain parameter blocks that do not fit dense-matrix geometry. One important case is the tied vo…
arXiv:2609.39595v1 Announce Type: new Abstract: Practical Muon maintains momentum and performs a small, fixed number of Newton--Schulz iterations separately for each parameter matrix, often with a Nesterov correction. We analyze these layer-wise finite-step updates jointly on a c…
arXiv:2609.39114v1 Announce Type: new Abstract: This paper examines how row-wise renormalization affects Muon, focusing on the gap between NorMuon's worst-case guarantees and its practical performance (Li et al.). Despite its growing adoption and promising performance in large la…
arXiv:2609.37525v1 Announce Type: new Abstract: Physical neural networks and analog in-memory computing could reduce the energy cost of neural network training. Realizing this potential, however, requires optimizers that combine effective learning with physical implementability. …
arXiv cs.AI
TIER_1English(EN)·Arthur C. B. de Oliveira, Dhruv D. Jatkar, Guilherme S. Vicinansa, Eduardo D. Sontag·
arXiv:2609.30546v1 Announce Type: cross Abstract: In this paper, we establish the first asymptotic convergence guarantees for the Muon algorithm through a more accurate proxy for the Newton-Schultz iteration than the typical matrix sign function. We prove that, for appropriate ch…
arXiv:2610.02158v1 Announce Type: cross Abstract: We consider the problem of sampling from Gibbs distributions on matrix spaces whose potential energies are neither convex nor globally gradient-Lipschitz. We introduce a family of non-quadratic kinetic energies that lead to a new …
<h3>Muon: The Orthogonalized Optimizer Explained Through Equations, Intuition, Code and Infographic</h3><h4>From SGD, to Adam’s per-coordinate scaling, to the five Newton–Schulz steps that turn a gradient into an orthogonal update</h4><p>For about a decade, the optimizer was the …
