Maksim Bolshim Antonovich

New method estimates neural network training curvature

Researchers have developed a novel stochastic estimator to calculate the trace of diagonal blocks of the Hessian matrix for neural networks. This method, which combines Hutchinson's estimator with a single Hessian-vecto…