New method rejuvenates LLM plasticity for better RL after SFT

Researchers have identified a phenomenon called "model plasticity loss" that hinders the effectiveness of Reinforcement Learning (RL) after Supervised Fine-Tuning (SFT) for large language models. Excessive SFT can lead to over-confident token distributions and difficult optimization landscapes, limiting RL's ability to further enhance model capabilities. To address this, a new method called "Rejuvenation" has been proposed, which uses base-anchored model fusion and targeted neuron resets to restore plasticity while retaining SFT benefits, showing improved performance on reasoning and agentic tasks. AI

IMPACT Addresses a key limitation in LLM training pipelines, potentially improving model performance on complex tasks.

RANK_REASON Academic paper proposing a new method for LLM training. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• Large Language Model
• Reinforcement Learning
• Rejuvenation
• Supervised Fine-Tuning

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →