English(EN) Limit Theorems for Stochastic Gradient Descent in High-Dimensional Single-Layer Networks

新研究完善了SGD泛化界限和协方差估计

作者 PulseAugur 编辑部 · [3 个来源] · 2026-04-23 01:48

研究人员开发了新的方法来分析机器学习中随机梯度下降（SGD）的泛化能力。一篇论文引入了可预测的历史自适应虚拟扰动，通过考虑依赖于优化历史的自适应噪声几何形状，从而实现更精确的泛化界限。另一项研究检查了单层网络中在线SGD的高维缩放极限，揭示了临界步长和信息指数如何影响样本复杂度和随机波动。 AI

影响这些对SGD理解的理论进展可能为未来的机器学习模型带来更强大、更有效的训练方法。

排序理由该集群包含两篇关于机器学习算法理论方面的学术论文。

AI 生成摘要 · Google Gemini · 来自 3 个来源。我们如何撰写摘要 →

报道来源 [3]

arXiv cs.LG TIER_1 English(EN) · Mohammad Partohaghighi · 2026-05-04 04:00

Information-Theoretic Generalization Bounds for Stochastic Gradient Descent with Predictable Virtual Noise

arXiv:2605.00064v1 Announce Type: new Abstract: Information-theoretic generalization bounds analyze stochastic optimization by relating expected generalization error to the mutual information between learned parameters and training data. Virtual perturbation analyses of SGD add a…
arXiv stat.ML TIER_1 English(EN) · Parsa Rangriz · 2026-05-01 04:00

Limit Theorems for Stochastic Gradient Descent in High-Dimensional Single-Layer Networks

arXiv:2511.02258v2 Announce Type: replace Abstract: This paper studies the high-dimensional scaling limits of online stochastic gradient descent (SGD). Building on the recent work of Ben Arous, Gheissari, and Jagannath on the effective dynamics of SGD, we study the critical scali…
arXiv stat.ML TIER_1 English(EN) · Wei Biao Wu · 2026-04-23 01:48

Refining Covariance Matrix Estimation in Stochastic Gradient Descent Through Bias Reduction

We study online inference and asymptotic covariance estimation for the stochastic gradient descent (SGD) algorithm. While classical methods (such as plug-in and batch-means estimators) are available, they either require inaccessible second-order (Hessian) information or suffer fr…