对使用SGD训练的两层ReLU神经网络进行的新的理论分析揭示，优化过程优先学习虚假相关性而非真实信号特征。研究表明，SGD可以呈指数级速度学习这些虚假特征，并且它们的存在会积极抑制真实信号的学习。该研究确定了学习动态中的特定相变，展示了特征和权重符号的一致性如何加速虚假学习，而大的裕度可以抑制信号学习。 AI
影响 强调了AI训练中的一个根本性挑战，表明当前的优化方法可能固有地偏向捷径，影响模型的可靠性和泛化能力。
排序理由 学术论文，详细阐述了神经网络训练动态的理论分析。[lever_c_demoted from research: ic=1 ai=1.0]
AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →
arXiv:2606.30444v1 Announce Type: new Abstract: Neural networks are known to be susceptible to over-reliance on spurious correlations. However, the precise mechanism by which models exploit shortcut features is not fully understood, and algorithms to mitigate this behavior rely o…
Neural networks are known to be susceptible to over-reliance on spurious correlations. However, the precise mechanism by which models exploit shortcut features is not fully understood, and algorithms to mitigate this behavior rely on as yet unjustified assumptions about the learn…