This paper establishes new theoretical bounds for nonexpansive stochastic approximation (SA) iterations, specifically focusing on the Krasnoselskii-Mann (KM) method. The research analyzes fixed-point mappings that are nonexpansive with respect to the Euclidean norm and utilizes polynomially decaying stepsizes. The findings detail optimal bounds under three distinct noise conditions, revealing that while some conditions match noiseless iterations, others necessitate square-summable stepsizes for convergence. The framework is applied to algorithms like stochastic gradient descent and learning in cocoercive games. AI
IMPACT Provides theoretical underpinnings for optimization algorithms used in machine learning, potentially improving training efficiency.
RANK_REASON Academic paper published on arXiv detailing theoretical advancements in stochastic approximation. [lever_c_demoted from research: ic=1 ai=0.7]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →