StableGrad

StableGrad stabilizes deep neural network training without batch normalization

Researchers have introduced StableGrad, a novel optimizer-level mechanism designed to control the scale of activations and gradients in deep neural networks. This method aims to prevent training instability without rely…