研究人员正在开发新方法来解决持续学习中的灾难性遗忘问题，即模型在学习新任务时会丢失先前获得的知识。几篇论文提出了新颖的技术，包括可学习的小波激活、技能引导的自适应记忆检索和持续蒸馏学习。其他方法侧重于表示微调、激活加权自适应保留和分组 LoRA 适配器合并，以提高模型的塑性和稳定性。这些进展旨在使模型能够更有效地进行顺序学习并适应不断变化的数据流。
AI
影响
持续学习领域的这些进展可能带来更强大、更具适应性的 AI 系统，这些系统能够在更长的时间内进行学习而不会出现性能下降。
arXiv:2407.13911v5 Announce Type: replace-cross Abstract: Prompt-based continual learning has shown strong performance in rehearsal-free class-incremental learning by adapting learnable prompts while freezing a pre-trained Vision Transformer (ViT) backbone. However, the effect of…
arXiv:2608.12874v1 Announce Type: new Abstract: Plasticity loss has emerged as a critical challenge in continual learning that significantly hinders the acquisition of sequential tasks. While optimizing activation designs offers a potential solution, current fixed-form functions …
arXiv:2608.12720v1 Announce Type: cross Abstract: While Large Language Model (LLM) agents increasingly rely on long-term memory for persistent interactions, the retrieval mechanisms governing this memory are rarely treated as evolvable components. This static approach limits perf…
Plasticity loss has emerged as a critical challenge in continual learning that significantly hinders the acquisition of sequential tasks. While optimizing activation designs offers a potential solution, current fixed-form functions suffer from an inherent spectral bias towards lo…
arXiv:2603.11201v3 Announce Type: replace-cross Abstract: The world is inherently dynamic, and continual learning aims to enable models to adapt to ever-evolving data streams. While pre-trained models have shown powerful performance in continual learning, they still require finet…
arXiv:2608.11758v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) exhibit strong generalization and reasoning abilities due to large-scale multimodal pre-training. However, fine-tuning these models on downstream tasks often leads to catastrophic forgetting,…
arXiv cs.LG
TIER_1English(EN)·Sergi Masip, Gido M. van de Ven, Javier Ferrando, Tinne Tuytelaars·
arXiv:2601.22012v3 Announce Type: replace Abstract: Catastrophic forgetting in continual learning is often measured at the performance or last-layer representation level, overlooking the underlying mechanisms. We introduce a mechanistic framework that offers a geometric interpret…
arXiv cs.LG
TIER_1English(EN)·Irene Testa, Luigi Quarantiello, Eric Nuertey Coleman, Samrat Mukherjee, Julio Hurtado, Vincenzo Lomonaco·
arXiv:2509.13211v4 Announce Type: replace Abstract: The ability to learn continuously over time remains a major challenge for modern machine learning systems, even in the era of Foundation Models. While the rich representations learned by large pre-trained models can partially mi…
arXiv cs.CL
TIER_1English(EN)·Mind Lab, :, Vin Bo, Asher Cai, Jingwei Cao, Song Cao, Vic Cao, Amelia Chen, Andrew Chen, Kaijie Chen, Cleon Cheng, Steven Chiang, Kaixuan Fan, Hera Feng, Huan Feng, Arthur Fu, Jun Gao, Pyke Han, Nolan Ho, Ori Hong, Hailee Hou, Piers Hua, Charles Huang,…·
arXiv:2608.09819v1 Announce Type: cross Abstract: Macaron-V1 is an open agent-model family for experiential intelligence: learning from experience in real environments and continuing to learn after deployment. It is organized around two system goals. Adaptation is pursued through…
arXiv cs.LG
TIER_1English(EN)·Idan Shenfeld, Mehul Damani, Jonas H\"ubotter, Pulkit Agrawal·
arXiv:2601.19897v2 Announce Type: replace Abstract: Continual learning, enabling models to acquire new skills and knowledge without degrading existing capabilities, remains a fundamental challenge for foundation models. While on-policy reinforcement learning can reduce forgetting…
Macaron-V1 is an open agent-model family for experiential intelligence: learning from experience in real environments and continuing to learn after deployment. It is organized around two system goals. Adaptation is pursued through recursive improvement of versioned model-harness …
arXiv cs.AI
TIER_1English(EN)·Zhiyan Hou, Dan Zhang, Tao Feng, Liyuan Wang, Wei Li, Xiangzhao Hao, Hongyan An, Junfeng Fang, Haokai Ma, Zhaohui Xu, Haiyun Guo, Jinqiao Wang, Tat-Seng Chua·
arXiv:2608.06216v1 Announce Type: cross Abstract: Classical continual learning (CL) has primarily focused on enabling models to update and retain knowledge through parameter-centric mechanisms, e.g., training strategies, architectural designs, and weight adaptation. However, emer…
arXiv:2608.04358v1 Announce Type: new Abstract: Continual learning (CL) requires models to learn tasks sequentially, yet deep neural networks often suffer from plasticity loss and poor knowledge transfer, which can impede their long-term adaptability. Drawing high-level inspirati…
arXiv:2604.02778v2 Announce Type: replace Abstract: Real-world multimodal knowledge graphs (MMKGs) are dynamic, with new entities, relations, and multimodal knowledge emerging over time. Existing continual knowledge graph reasoning (CKGR) methods focus on structural triples and c…
arXiv cs.AI
TIER_1English(EN)·R. Blake Lawlor, Daniel S. Brown·
arXiv:2608.04334v1 Announce Type: cross Abstract: Contemporary model-free reinforcement learning algorithms can achieve very high performance, but have low sample efficiency and are not robust to changes in the environment. Model-based algorithms have much higher sample efficienc…
Classical continual learning (CL) has primarily focused on enabling models to update and retain knowledge through parameter-centric mechanisms, e.g., training strategies, architectural designs, and weight adaptation. However, emerging paradigms are reshaping the scope of CL beyon…
A recurrent Transformer with fixed-size memory and coupled prefiller-decoder training improves long-context modeling while enabling efficient parallel training and reduced inference cost.
arXiv cs.LG
TIER_1English(EN)·Jeong Min Kong, Richard S. Sutton·
arXiv:2608.01475v1 Announce Type: new Abstract: Neural networks that can grow or both grow and shrink during learning, referred to as growing neural networks and elastic neural networks, respectively, have recently been explored in offline continual learning with a particular foc…
arXiv:2608.00630v1 Announce Type: new Abstract: Achieving continual learning (CL) with deep neural networks requires balancing stability and plasticity while enabling knowledge transfer. In this work, we focus on offline learning algorithms under the constraints: (I) no access to…
arXiv:2607.28663v1 Announce Type: cross Abstract: Artificial Intelligence (AI) systems often perform well on isolated tasks but struggle under continual learning conditions, where training on new tasks can overwrite previously acquired knowledge, a failure mode known as catastrop…
We introduce CMP (Cognitive Memory Primitive), a continual-learning architecture that repre?sents inputs as sparse relational codes, stores them in a two-tier competitive memory, and learns through local updates without end-to-end backpropagation through its feature-generating sy…
Artificial Intelligence (AI) systems often perform well on isolated tasks but struggle under continual learning conditions, where training on new tasks can overwrite previously acquired knowledge, a failure mode known as catastrophic forgetting. Biological learning systems reduce…
arXiv:2608.11690v1 Announce Type: cross Abstract: Continual learning must absorb new tasks without erasing old ones, and replay---mixing a small buffer of past examples into current training---is among the most effective remedies for catastrophic forgetting. Yet its generalizatio…
<img alt="" src="https://res.cloudinary.com/lesswrong-2-0/image/upload/v1786558686/lexical_client_uploads/nxutnane7qutvvsgxbpl.webp" /><p><span>If you've ever screamed in all-caps at an AI, then you know the difference between what it learned when it was trained, and what you can…
arXiv:2608.01314v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) increasingly rely on long chain-of-thought reasoning for complex tasks. However, as reasoning sequences lengthen, models may gradually rely less on visual evidence and more on accumulated tex…
