Research: RL better preserves LLM circuits than SFT, reducing catastrophic forgetting

A new research paper explores the phenomenon of catastrophic forgetting in large language models, specifically comparing reinforcement learning (RL) and supervised fine-tuning (SFT). The study found that while SFT adapts more quickly to new tasks, it causes significant disruption to the model's internal circuits and leads to greater forgetting of prior capabilities. In contrast, RL preserves more of the original model's circuits, albeit with slower task adaptation, suggesting this circuit preservation is key to RL's robustness against catastrophic forgetting. AI

IMPACT This research suggests that RL-based fine-tuning may be a more stable method for adapting LLMs without sacrificing existing knowledge.

RANK_REASON The cluster contains an academic paper detailing a new mechanistic analysis of model behavior. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• catastrophic forgetting
• Large Language Models
• Qwen2.5-3B-Instruct
• Reinforcement Learning
• scientific question-answering
• Supervised Fine-Tuning

• paper
• safety

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →