English(EN) Catastrophic Overfitting is Weight Drift the Training Loss Cannot See

已识别并解决人工智能训练中的灾难性过拟合问题

作者 PulseAugur 编辑部 · [1 个来源] · 2026-11-05 05:00

研究人员在神经网络的快速对抗性训练（FAT）中识别出一种称为灾难性过拟合（CO）的现象。当对多步攻击的鲁棒性迅速下降时，即使网络在单步攻击和干净数据上的性能持续提高，也会发生 CO。这是因为训练损失对这种权重漂移基本不敏感。该研究提出了一种简单的解决方案：使用权重的指数移动平均值（EMA），该平均值保持鲁棒性，并在验证鲁棒性下降时将权重重置为其 EMA。此方法可有效消除 CO，并在 CIFAR-100 等基准数据集上实现最先进的准确度，同时降低了训练成本。 AI

影响识别出一种关键的训练故障模式，并提供了一种实用的、低成本的解决方案来提高模型鲁棒性。

排序理由详细介绍机器学习训练中新发现和拟议解决方案的学术论文。[lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.LG 阅读 →

AI 生成摘要 · Google Gemini · 来自 1 个来源。我们如何撰写摘要 →

本文如何被排名

Signal score: 26 / 100; Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket: Tool; 详细介绍机器学习训练中新发现和拟议解决方案的学术论文。[lever_c_demoted from research: ic=1 ai=1.0]
Source corroboration: Single-source cluster; Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics: paper, safety; Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance: High; Clearly on-topic for AI-industry coverage.
Story freshness: Breaking (< 6h); Fresh story with cross-source coverage still developing. Ranking may shift as more sources report.

完整方法见我们的编辑标准。

报道来源 [1]

arXiv cs.LG TIER_1 English(EN) · Chao Pan · 2026-11-05 05:00

灾难性过拟合是训练损失无法察觉的权重漂移

arXiv:2611.06082v1 Announce Type: new Abstract: Fast adversarial training (FAT) makes robust training affordable by replacing the multi-step attack with a single-step one, but it suffers from catastrophic overfitting (CO): partway through training, robustness to multi-step attack…

本文如何被排名

报道来源 [1]

灾难性过拟合是训练损失无法察觉的权重漂移

相关实体

相关话题