New optimization method adapts to unknown smoothness conditions

Researchers have developed a novel two-phase optimization method designed to improve convergence rates for stochastic non-convex optimization. This method adapts to different smoothness conditions (average, standard, and noiseless) without requiring prior knowledge of smoothness constants or variance bounds. The approach utilizes a normalized momentum strategy when gradient differences exceed a threshold, ensuring bounded expected function gaps and achieving rates of O(T^{-1/3}) under average smoothness, O(T^{-1/4}) under standard smoothness, and O(T^{-1/2}) in noiseless settings. AI

IMPACT This method could improve the efficiency of training machine learning models by optimizing their underlying algorithms.

RANK_REASON The item is an academic paper detailing a new optimization method. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• AdaGrad-Norm
• Adaptive Variance Reduction without Knowing Oracle Smoothness
• alphaXiv
• arXiv
• CatalyzeX Code Finder for Papers
• Connected Papers
• CORE Recommender
• DagsHub
• Gotit.pub
• Hugging Face
• Influence Flower
• Litmaps
• ScienceCast
• scite Smart Citations

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →