三篇新研究论文探讨了持续学习系统中灾难性遗忘的现象，特别是在大型语言模型中。第一篇论文引入了一个受控框架来研究遗忘机制，提出表示强度和特征稀疏性起着关键作用，而不仅仅是叠加。第二篇和第三篇论文（似乎是相同的）在神经切线核（NTK）框架下提供了一个函数空间理论，提出遗忘是低秩的，并集中在特定的输出空间方向。第四篇论文对二十个最先进模型进行了机制分析，识别了脆弱的神经回路，并引入了一种名为低秩电路投影（LRCP）的新干预措施来减轻遗忘。
AI
影响
这些研究为提高 AI 模型在持续学习和适应过程中的稳定性和性能提供了新的理论框架和实用方法。
排序理由
该集群包含多篇在 arXiv 上发表的学术论文，详细介绍了对 AI 模型行为的理论和实证研究。
arXiv:2606.20431v1 Announce Type: new Abstract: Continual learning (CL) systems often forget previously acquired knowledge, yet the mechanisms driving forgetting remain hard to isolate in practice because real datasets entangle many factors. We present a controlled, toy-world fra…
Continual learning (CL) systems often forget previously acquired knowledge, yet the mechanisms driving forgetting remain hard to isolate in practice because real datasets entangle many factors. We present a controlled, toy-world framework that makes these mechanisms observable an…
arXiv cs.AI
TIER_1English(EN)·Ido Nitzan Hidekel, Dan Raviv·
arXiv:2606.18024v1 Announce Type: cross Abstract: Catastrophic forgetting in continual adaptation is usually studied through parameter drift, replay, or distillation, but these views do not identify which output-space directions are vulnerable. We give a function-space account in…
Catastrophic forgetting in continual adaptation is usually studied through parameter drift, replay, or distillation, but these views do not identify which output-space directions are vulnerable. We give a function-space account in the NTK regime: new-task training induces old-tas…
arXiv cs.CL
TIER_1English(EN)·Gustav Olaf Yunus Laitinen-Fredriksson Lundstrom-Imanov·
arXiv:2601.18699v2 Announce Type: replace-cross Abstract: Sequential fine-tuning of Large Language Models (LLMs) adaptation to target tasks often triggers catastrophic forgetting, where the acquisition of novel target skills degrades ancestral capabilities. This paper presents a …
