Alberto Fernández-Hernández
PulseAugur coverage of Alberto Fernández-Hernández — every cluster mentioning Alberto Fernández-Hernández across labs, papers, and developer communities, ranked by signal.
1 day(s) with sentiment data
-
Adam optimizer's tied momentum parameters reveal unique training dynamics
A new paper published on arXiv explores the dynamics of the Adam optimizer, a core component in large-scale AI training. The research identifies a specific mechanism related to its two momentum parameters, $\beta_1$ and…
-
New OUIDecay method adapts CNN regularization layer-by-layer
Researchers have introduced OUIDecay, a novel adaptive weight decay method for convolutional neural networks. This technique dynamically adjusts regularization strength for each layer based on online activation patterns…
-
New research refines Adam optimizer's memory and noise dynamics
Two new research papers explore the nuances of the Adam optimizer, a popular tool in deep learning. The first paper proposes a "refresh rule" for Adam's momentum parameter, suggesting it should scale with training data …