gradient clipping
PulseAugur coverage of gradient clipping — every cluster mentioning gradient clipping across labs, papers, and developer communities, ranked by signal.
1 day(s) with sentiment data
-
Four arXiv papers advance stochastic optimization theory · 4 sources tracked
Four new research papers published on arXiv explore advanced convergence properties of stochastic optimization methods. The first paper introduces a unified theory for steady-state convergence of stochastic approximatio…
-
New gradient descent method shows exponential convergence for over-parameterized models
Researchers have introduced Dual Space Preconditioned Gradient Descent, a novel method for optimizing models in over-parameterized regimes. This approach, which includes variations like Normalized Gradient Descent and G…
-
Gradient leakage attacks threaten GNNs in circuit design
A new research paper details the first comprehensive evaluation of gradient leakage attacks (GLAs) on graph neural networks (GNNs) used in circuit design and hardware security. The study reveals that GLAs can expose sen…
-
GradientStabilizer improves LLM training stability by controlling gradient norms
Researchers have developed GradientStabilizer, a new method to improve training stability in deep learning systems, particularly for large language models. This technique addresses issues caused by rare but extreme grad…