learning rate
PulseAugur coverage of learning rate — every cluster mentioning learning rate across labs, papers, and developer communities, ranked by signal.
2 day(s) with sentiment data
-
New theory reframes optimizer stability in deep learning
Researchers have identified a phenomenon in deep learning where gradient-based optimizers maintain stable Hessian eigenvalues above theoretically predicted instability thresholds. This deviation, observed to be as high …
-
Research links learning rates to catastrophic overtraining in LLMs
A new research paper explores how learning rates impact catastrophic overtraining during the supervised fine-tuning (SFT) of large language models (LLMs). The study, published on arXiv, suggests that different learning …
-
New Bayesian Framework Optimizes Neural Network Learning Rates
Researchers have introduced a novel probabilistic framework to optimize the learning rate in neural network training, moving beyond empirical trial-and-error. This new approach develops classic Bayesian statistics into …
-
Researchers pinpoint origin of neural network 'Edge of Stability' phenomenon
Researchers have introduced a new concept called the 'edge coupling' to explain the phenomenon known as the Edge of Stability in neural network training. This functional, applied to consecutive iterate pairs, helps to e…