Scion method

New theory advances scale-invariant neural network optimization

Researchers have developed new theoretical insights into optimizing neural networks, particularly concerning scale-invariant methods and heavy-tailed noise. They established a dimension-dependent lower bound for scale-i…