English(EN) Measuring the local shape of learning-rate transfer curves

新方程量化神经网络中学习率迁移曲线的梯度

作者 PulseAugur 编辑部 · [1 个来源] · 2026-10-16 04:00

研究人员开发了新的方法来定量表征神经网络中学习率（LR）迁移曲线的形状。他们提出了两个关系方程，用以经验性地解释LR迁移曲线的梯度，即LR梯度。其中一个方程是适用于稳定性边缘的理论公式，另一个是通过将梯度视为超参数梯度推导出的近似表达式。这些方程可以为曲线梯度设定下限，并识别归一化层中零梯度的条件，为理解LR依赖性奠定基础。 AI

影响为理解和优化神经网络训练动力学提供了理论基础。

排序理由该集群包含一篇详细介绍机器学习新理论和经验发现的学术论文。[lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.LG 阅读 →

AI 生成摘要 · Google Gemini · 来自 1 个来源。我们如何撰写摘要 →

本文如何被排名

Signal score: 14 / 100; Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket: Tool; 该集群包含一篇详细介绍机器学习新理论和经验发现的学术论文。[lever_c_demoted from research: ic=1 ai=1.0]
Source corroboration: Single-source cluster; Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics: paper, model release; Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance: High; Clearly on-topic for AI-industry coverage.
Story freshness: Breaking (< 6h); Fresh story with cross-source coverage still developing. Ranking may shift as more sources report.

完整方法见我们的编辑标准。

报道来源 [1]

arXiv cs.LG TIER_1 English(EN) · Satoki Ishikawa, Rio Yokota, Ryo Karakida · 2026-10-16 04:00

测量学习率迁移曲线的局部形状

arXiv:2610.21814v1 Announce Type: new Abstract: Understanding the learning-rate (LR) dependence of trained model performance is essential for better optimization of neural networks, but the appropriate size of the LR depends on learning dynamics, architecture, and their hidden bi…

本文如何被排名

报道来源 [1]

测量学习率迁移曲线的局部形状

相关实体

相关话题