New PASs-MoE method improves continual learning in LLMs

By PulseAugur Editorial · [1 sources] · 2026-07-20 04:00

Researchers have developed a new method called PASs-MoE to improve continual learning in multimodal large language models (MLLMs). This technique addresses the issue of "Misaligned Co-drift," where the model's router and experts deviate from their specialized tasks, leading to forgetting. PASs-MoE utilizes pathway activation subspaces to guide routing and stabilize important rank directions, enhancing accuracy and reducing forgetting without increasing model parameters. Experiments on a continual instruction tuning benchmark demonstrated superior performance compared to existing methods. AI

IMPACT Enhances LLM adaptability and reduces forgetting in continual learning scenarios.

RANK_REASON The cluster contains a research paper detailing a new method for continual learning in LLMs. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →

New PASs-MoE method improves continual learning in LLMs

COVERAGE [1]

arXiv cs.AI TIER_1 English(EN) · Zhiyan Hou, Haiyun Guo, Haokai Ma, Yandu Sun, Yonghui Yang, Jinqiao Wang · 2026-07-20 04:00

PASs-MoE: Mitigating Misaligned Co-drift among Router and Experts via Pathway Activation Subspaces for Continual Learning

arXiv:2601.13020v2 Announce Type: replace-cross Abstract: Continual instruction tuning (CIT) requires multimodal large language models (MLLMs) to adapt to a stream of tasks without forgetting prior capabilities. A common strategy is to isolate updates by routing inputs to differe…

COVERAGE [1]

PASs-MoE: Mitigating Misaligned Co-drift among Router and Experts via Pathway Activation Subspaces for Continual Learning

RELATED ENTITIES

RELATED TOPICS