Edge of Stability
PulseAugur coverage of Edge of Stability — every cluster mentioning Edge of Stability across labs, papers, and developer communities, ranked by signal.
2 day(s) with sentiment data
-
Logistic Regression Transition Time Bounds Disproven
Researchers have disproven a conjecture regarding the transition time bounds for separable logistic regression under gradient descent with large constant step sizes. Previously, it was believed that the transition time …
-
New research explores geometric optimization at the 'Edge of Stability' in associative memories
Researchers have explored the geometric properties of high-capacity kernel logistic regression (KLR) associative memories, identifying a critical hyperparameter regime known as the "Ridge of Optimization." This region i…
-
Federated Optimization: Edge of Stability Hinders SCAFFOLD Algorithm
A new research paper proposes that the "Edge of Stability" (EoS) phenomenon hinders the performance of the SCAFFOLD algorithm in federated optimization. Despite SCAFFOLD's theoretical advantages in handling data heterog…
-
New 'weight-norm criticality' explains AI training instability
Researchers have identified a new critical factor in deep neural network training instability, termed 'weight-norm criticality.' This phenomenon, distinct from the commonly understood 'learning-rate criticality,' arises…
-
Research paper reframes gradient descent as dynamical system
A new research paper explores finite-step gradient descent not as a simple optimization tool, but as a discrete dynamical system. The study analyzes how the training map's behavior, including phenomena like edge-of-stab…
-
New research frameworks model gradient descent at the edge of stability
Two new research papers explore the phenomenon of gradient descent operating at the edge of stability (EoS) in deep learning. The first paper introduces 'Edge Flow,' a system of differential equations that models gradie…
-
Research: Edge of Stability selectively shapes ML learning
A new research paper explores the "edge of stability" (EoS) in machine learning optimization, revealing it's not a uniform property but a selective one. The study demonstrates that EoS can redistribute learning across d…
-
Researchers pinpoint origin of neural network 'Edge of Stability' phenomenon
Researchers have introduced a new concept called the 'edge coupling' to explain the phenomenon known as the Edge of Stability in neural network training. This functional, applied to consecutive iterate pairs, helps to e…