PulseAugur
EN
LIVE 06:46:51

AdaGrad optimization algorithm fails on composite objectives, study finds

By PulseAugur Editorial · [2 sources] ·

A new paper demonstrates that the AdaGrad optimization algorithm does not adapt to Hölder-smoothness for composite objectives. The research highlights a specific convex composite optimization problem where AdaGrad fails to achieve the expected convergence rate. This occurs because the gradient of the smooth term may not vanish at the optimum, leading AdaGrad to excessively reduce its stepsize and slow down convergence. The paper also suggests alternative accumulation mechanisms that avoid this issue. AI

RANK_REASON Academic paper detailing a theoretical limitation of an optimization algorithm. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv stat.ML →

AI-generated summary · Google Gemini · from 2 sources. How we write summaries →

AdaGrad optimization algorithm fails on composite objectives, study finds

COVERAGE [2]

  1. arXiv stat.ML TIER_1 English(EN) · Matia Bojovic, Saverio Salzo, Massimiliano Pontil ·

    AdaGrad does not adapt to H\"older-smoothness for composite objectives

    arXiv:2606.29893v1 Announce Type: cross Abstract: We exhibit a simple deterministic one-dimensional convex composite optimization problem for which AdaGrad scheme does not achieve the classical convergence rate $\mathcal{O}(n^{-(1+\nu)/2})$ associated with H\"older-smooth objecti…

  2. arXiv stat.ML TIER_1 English(EN) · Massimiliano Pontil ·

    AdaGrad does not adapt to Hölder-smoothness for composite objectives

    We exhibit a simple deterministic one-dimensional convex composite optimization problem for which AdaGrad scheme does not achieve the classical convergence rate $\mathcal{O}(n^{-(1+ν)/2})$ associated with Hölder-smooth objectives. The example highlights a basic mismatch between c…

RELATED TOPICS