New theory precisely analyzes SGD stepsizes for convex optimization

This paper presents an exact worst-case analysis for stochastic gradient descent (SGD) in smooth convex optimization, focusing on the expected suboptimality of the final iterate. The research provides a matching objective function and noise distributions to demonstrate its theoretical guarantees for any schedule of N stepsizes. When the variance bound is sufficiently large, the paper identifies a unique, monotonically decreasing, minimax-optimal stepsize schedule for SGD, which differs from accelerated schedules seen in deterministic settings. AI

IMPACT Provides theoretical underpinnings for optimization algorithms used in machine learning.

RANK_REASON Academic paper on optimization theory. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• arXiv
• Convex Minimization with Constraints of Systems of Variational Inequalities, Mixed Equilibrium, Variational Inequality, and Fixed Point Problems
• Hugging Face
• stochastic gradient descent

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →