两篇新研究论文探讨了机器学习优化算法的进展。一篇论文对Adam优化器进行了理论分析，详细说明了其在非平稳目标下的性能，并确定了噪声和漂移之间的权衡。第二篇论文通过引入小批量收敛性分析和混合切换策略（包括抖动和向SGD的过渡）来增强SignSGD算法，在图像分类任务上实现了具有竞争力的准确性。 AI
影响 这些论文为优化器提供了理论见解和实际改进，有望实现更高效、更准确的机器学习模型训练。
排序理由 两篇在arXiv上发表的学术论文，对机器学习优化器进行了理论分析和算法增强。
AI 生成摘要 · Google Gemini · 来自 4 个来源。 我们如何撰写摘要 →
arXiv:2605.04269v1 Announce Type: cross Abstract: We provide a theoretical analysis of Adam under non-stationary stochastic objectives, separating two regimes: Euclidean tracking under adaptive strong monotonicity of the Adam-preconditioned mean-gradient operator, and high-probab…
arXiv:2604.25550v1 Announce Type: new Abstract: SignSGD compresses each stochastic gradient coordinate to a single bit, offering substantial memory and communication savings, but its 1-bit quantization removes magnitude information and is known to leave a generalization gap relat…
SignSGD compresses each stochastic gradient coordinate to a single bit, offering substantial memory and communication savings, but its 1-bit quantization removes magnitude information and is known to leave a generalization gap relative to well-tuned SGD. We revisit SignSGD from a…
We provide a theoretical analysis of Adam under non-stationary stochastic objectives, separating two regimes: Euclidean tracking under adaptive strong monotonicity of the Adam-preconditioned mean-gradient operator, and high-probability projected stationarity guarantees under gene…