Schattor: Schatten-family methods for deep learning optimization
Researchers have introduced Schattor, a new family of adaptive optimization methods for deep learning that utilize Schatten norms. This framework unifies existing methods like SGD and Muon, addressing challenges posed by complex parameter structures and noisy gradients in modern deep learning. Schattor aims to provide theoretical guarantees for stochastic matrix optimization problems and includes extensions for multi-block optimization. AI