cosine annealing
PulseAugur coverage of cosine annealing — every cluster mentioning cosine annealing across labs, papers, and developer communities, ranked by signal.
-
New WSqD learning rate schedule offers horizon-free training for large models
Researchers have introduced WSqD, a novel learning rate schedule designed for large model training that is independent of the training horizon. Unlike existing methods like cosine annealing and Warmup-stable-decay (WSD)…
-
New CWGD method improves optimization noise measurement for deep learning
Researchers have developed a new method called Curvature-Weighted Gradient Diversity (CWGD) to better measure optimization noise in deep learning models. Unlike traditional methods that treat all parameter directions eq…
-
New Transformer Model Enhances Stock Index Forecasting with Data Augmentation
Researchers have developed a new Transformer-based architecture for one-step stock index forecasting, addressing challenges like noisy signals and distributional shifts in financial time series. The proposed framework i…