Shuai Jiang

Paper explores preconditioned gradient descent's impact on neural network learning regimes

This paper investigates how preconditioned gradient descent (PGD) methods, like Gauss-Newton, influence spectral bias and the phenomenon of grokking in neural networks. Researchers propose that PGD can mitigate spectral…