研究人员开发了一种名为逐样本裁剪随机梯度下降（PS-Clip-SGD）的新训练方法，该方法提高了非凸优化问题的鲁棒性和速度。该方法为收敛提供了理论保证，即使存在重尾梯度噪声。实证测试表明，在CIFAR-100上训练AlexNet时，PS-Clip-SGD的性能优于标准技术，并且在与梯度累积结合使用时也显示出优势。 AI
影响 引入了一种新颖的训练技术，可能带来更高效、更稳定的模型开发。
排序理由 详细介绍机器学习新优化方法的学术论文。
AI 生成摘要 · Google Gemini · 来自 3 个来源。 我们如何撰写摘要 →
We propose a robust gradient estimator based on per-sample gradient clipping and analyze its properties both theoretically and empirically. We show that the resulting method, per-sample clipped SGD (PS-Clip-SGD), achieves optimal in-expectation convergence rates for non-convex op…
arXiv:2605.02701v1 Announce Type: cross Abstract: We propose a robust gradient estimator based on per-sample gradient clipping and analyze its properties both theoretically and empirically. We show that the resulting method, per-sample clipped SGD (PS-Clip-SGD), achieves optimal …
We propose a robust gradient estimator based on per-sample gradient clipping and analyze its properties both theoretically and empirically. We show that the resulting method, per-sample clipped SGD (PS-Clip-SGD), achieves optimal in-expectation convergence rates for non-convex op…