Hessian
PulseAugur coverage of Hessian — every cluster mentioning Hessian across labs, papers, and developer communities, ranked by signal.
5 day(s) with sentiment data
-
Genetic algorithms mimic clipped gradient descent in high-dimensional AI search
Researchers have demonstrated that genetic algorithms can effectively function as a form of clipped gradient descent in high-dimensional search spaces. This process involves mutation-selection mechanisms that implicitly…
-
New metric CWGD improves optimization noise measurement in ML
Researchers have introduced Curvature-Weighted Gradient Diversity (CWGD), a novel metric designed to better measure optimization noise in machine learning models. Unlike traditional methods that treat all parameter dire…
-
New method offers second-order KKT guarantees for Bregman ADMM
Researchers have developed a novel approach to analyze Bregman ADMM for nonconvex and non-Lipschitz optimization problems. This method replaces the standard Lipschitz gradient assumption with a two-sided relative smooth…
-
New research frameworks model gradient descent at the edge of stability
Two new research papers explore the phenomenon of gradient descent operating at the edge of stability (EoS) in deep learning. The first paper introduces 'Edge Flow,' a system of differential equations that models gradie…
-
New 'Architecture Warm-Up' Stabilizes Transformer Training
Researchers have developed a new method to stabilize the training of large Transformer models, which are often prone to instability and divergence. The approach, called "architecture warm-up," involves progressively inc…
-
New optimization techniques emerge for faster, more efficient AI model training · 8 sources tracked
Several recent arXiv papers explore advancements in optimization techniques for machine learning. Researchers have proposed new methods like Weight Adaptation ASNG (WA-ASNG) to improve parallel performance in evolutiona…
-
New research explores how network symmetry aids optimization in overparameterized deep learning models.
A new paper analyzes how overparameterization in neural networks aids optimization by introducing additional symmetries. These symmetries act as a form of preconditioning on the Hessian, leading to better-conditioned mi…