Four new research papers published on arXiv explore advanced convergence properties of stochastic optimization methods. The first paper introduces a unified theory for steady-state convergence of stochastic approximation with contractive, Markovian, and multiplicative noise, achieving an optimal $O(\sqrt{\alpha})$ rate in Wasserstein-2 distance. Another paper proves almost sure convergence for stochastic gradient descent with clipping and additive Gaussian noise, extending to momentum variants. A third study analyzes convergence rates for SGD under heavy-tailed noise and Hölder smoothness, introducing new guarantees for gradient clipping methods in challenging regimes. Finally, a fourth paper presents a method using random tilts to find stationary points in stochastic convex optimization, achieving a $\sqrt{d/n}$ stationarity residual order. AI
IMPACT Advances theoretical understanding of optimization algorithms crucial for training large-scale AI models.
RANK_REASON Cluster consists of four academic papers on theoretical machine learning topics published on arXiv.
- arXiv
- $\delta$-GClip
- Gaussian noise
- G-Clip
- gradient clipping
- heavy-tailed noise
- momentum variants
- SGD
- stochastic gradient descent
- alphaXiv
- CatalyzeX
- CORE Recommender
- DagsHub
- Gaussian function
- Gotit.pub
- Hugging Face
- IArxiv Recommender
- Influence Flower
- ScienceCast
- Steady-State Convergence
- stochastic approximation
AI-generated summary · Google Gemini · from 5 sources. How we write summaries →
