Alberto Fernández-Hernández
PulseAugur coverage of Alberto Fernández-Hernández — every cluster mentioning Alberto Fernández-Hernández across labs, papers, and developer communities, ranked by signal.
-
New method optimizes Adam optimizer memory parameter \beta
Researchers have developed a novel method for optimizing the memory parameter \beta in the Adam optimizer. This technique involves a short pilot training phase to select an optimal \beta value, which is then fixed for t…
-
Adam optimizer's tied momentum parameters reveal unique training dynamics
A new paper published on arXiv explores the dynamics of the Adam optimizer, a core component in large-scale AI training. The research identifies a specific mechanism related to its two momentum parameters, $\beta_1$ and…
-
New OUIDecay method adapts CNN regularization layer-by-layer
Researchers have introduced OUIDecay, a novel adaptive weight decay method for convolutional neural networks. This technique dynamically adjusts regularization strength for each layer based on online activation patterns…
-
New research refines Adam optimizer's memory and noise dynamics
Two new research papers explore the nuances of the Adam optimizer, a popular tool in deep learning. The first paper proposes a "refresh rule" for Adam's momentum parameter, suggesting it should scale with training data …