Gemma 3 1B-style

New optimizers respect neural network symmetries, improve training

Researchers have introduced a new principle for designing optimizers in deep learning that aligns with the inherent symmetries of neural network architectures. Unlike current optimizers like Adam, which operate on param…