English(EN)Introducing Nested Learning: A new ML paradigm for continual learning
AI持续学习研究应对灾难性遗忘
作者PulseAugur 编辑部·[18 个来源]·
研究人员正在探索AI持续学习的新方法，旨在克服“灾难性遗忘”的挑战，即模型在学习新技能时会丢失先前学到的信息。Google Research推出了“嵌套学习”，一种将模型视为相互关联的优化问题的范式，以缓解此问题。其他研究侧重于高效方法，如CIRCLE，它使用固定的存储库特征，以及用于LLM的SAE引导激活正则化，它在激活空间而非权重空间中运行。此外，正在开发新的指标来更好地表征遗忘，并提出了CoVON等新型优化器来平衡持续学习系统的稳定性和可塑性。调查结果表明，AI安全研究人员在广泛的持续学习代理的未来时间表和风险方面缺乏共识。
AI
arXiv:2607.00377v1 Announce Type: new Abstract: Self-supervised Continual Graph Learning (CGL) aims to successively learn from a graph sequence with different tasks without label supervision - a paradigm that has attracted widespread attention. Most existing self-supervised CGL m…
arXiv cs.AI
TIER_1English(EN)·Julien Lefebvre, Stefan Duffner, Mathieu Lefort·
arXiv:2606.31275v1 Announce Type: cross Abstract: Online Continual Self-Supervised Learning (OCSSL) aims to learn representations from a continuous stream of unlabeled data, without knowledge of task boundaries and under memory constraints. Existing methods rely either on replay …
Self-supervised Continual Graph Learning (CGL) aims to successively learn from a graph sequence with different tasks without label supervision - a paradigm that has attracted widespread attention. Most existing self-supervised CGL methods rely on instance-level consistency object…
arXiv:2605.12998v3 Announce Type: replace Abstract: Continual graph learning (CGL) aims to learn from dynamically evolving graphs while mitigating catastrophic forgetting. Existing CGL approaches typically adopt a task-based formulation, where the data stream is partitioned into …
arXiv:2606.26629v1 Announce Type: cross Abstract: Weight-space regularization methods such as Elastic Weight Consolidation (EWC) are the standard approach to catastrophic forgetting in continual learning. However, those methods tend to underperform when applied to large language …
arXiv:2606.27095v1 Announce Type: cross Abstract: Cold-start exemplar-free class-incremental learning requires learning a growing set of classes without replay, external pretraining, or a large initial task. Existing cold-start methods typically either train the backbone througho…
Cold-start exemplar-free class-incremental learning requires learning a growing set of classes without replay, external pretraining, or a large initial task. Existing cold-start methods typically either train the backbone throughout the stream and compensate for semantic drift, o…
Weight-space regularization methods such as Elastic Weight Consolidation (EWC) are the standard approach to catastrophic forgetting in continual learning. However, those methods tend to underperform when applied to large language models. We argue that such underperformance can be…
arXiv:2512.18471v2 Announce Type: replace Abstract: Continual learning systems face a fundamental geometric obstacle: as experience accumulates on a fixed-capacity manifold, covering numbers grow linearly with time, eventually forcing representational overlap and catastrophic int…
arXiv cs.LG
TIER_1English(EN)·Ahmed Anwar, Andreas Wagner, Federico Raue, Tobias Nauen, Andreas Dengel·
arXiv:2606.25165v1 Announce Type: new Abstract: Accuracy degradation is the standard metric for Catastrophic Forgetting (CF), however, it records only whether forgetting occurred or not. It saturates at the extremes and collapses discretely at task boundaries, hiding the internal…
arXiv cs.AI
TIER_1English(EN)·Subarnaduti Paul, Yohan Jung, Mohammad Emtiyaz Khan, Siddharth Swaroop, Thomas M\"ollenhoff, Martin Mundt·
arXiv:2606.24007v1 Announce Type: cross Abstract: Continual learning remains a major challenge for modern deep networks, partly because commonly used optimizers lack inherent mechanisms for continual adaptation. One such natural mechanism is fast and slow adaptation to balance st…
Accuracy degradation is the standard metric for Catastrophic Forgetting (CF), however, it records only whether forgetting occurred or not. It saturates at the extremes and collapses discretely at task boundaries, hiding the internal structure of what is being forgotten. We introd…
Continual learning remains a major challenge for modern deep networks, partly because commonly used optimizers lack inherent mechanisms for continual adaptation. One such natural mechanism is fast and slow adaptation to balance stability and plasticity. This mechanism has deep ro…
arXiv stat.ML
TIER_1English(EN)·Matan Schliserman, Gon Buzaglo, Itay Evron, Daniel Soudry·
arXiv:2606.30559v1 Announce Type: cross Abstract: We characterize weakly regularized continual classification in homogeneous models as sequential projections onto task margin sets. This result generalizes prior analyses restricted to either stationary (single-task) deep models or…
We characterize weakly regularized continual classification in homogeneous models as sequential projections onto task margin sets. This result generalizes prior analyses restricted to either stationary (single-task) deep models or continual linear models. We show that global conv…
<p><i><span>This is the fifth post in the sequence </span></i><a href="https://www.lesswrong.com/s/oc5Auteiibo56kNXw"><i><span>Implications of Continual Learning for LLM Agents</span></i></a><i><span>.</span></i></p><h1><span>Summary</span></h1><p><span>While writing our continua…
<!-- SC_OFF --><div class="md"><p>My question on live continual learning use cases was removed by moderators here because they think i asked basic level question about live continual learning which i thought is a frontier level research. But anyways. Is anyone interested in talki…
