Tensorion

Tensorion optimizer generalizes Muon for tensor-aware machine learning

Researchers have introduced Tensorion, a novel tensor-aware optimizer that generalizes the Muon optimizer by considering higher-order tensors. Unlike common optimizers that treat parameter blocks as unstructured vectors…