一篇新论文表明，AdaGrad 优化算法不适应复合目标的 Hölder 光滑性。研究人员指出了一个特定的凸复合优化问题，在该问题上 AdaGrad 未能达到预期的收敛速率。这是因为光滑项的梯度在最优值处可能不会消失，导致 AdaGrad 过度减小其步长并减慢收敛速度。该论文还提出了避免此问题的替代累积机制。 AI
排序理由 学术论文，详细说明了优化算法的理论局限性。[lever_c_demoted from research: ic=1 ai=1.0]
AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →
arXiv:2606.29893v1 Announce Type: cross Abstract: We exhibit a simple deterministic one-dimensional convex composite optimization problem for which AdaGrad scheme does not achieve the classical convergence rate $\mathcal{O}(n^{-(1+\nu)/2})$ associated with H\"older-smooth objecti…
We exhibit a simple deterministic one-dimensional convex composite optimization problem for which AdaGrad scheme does not achieve the classical convergence rate $\mathcal{O}(n^{-(1+ν)/2})$ associated with Hölder-smooth objectives. The example highlights a basic mismatch between c…