New GIFT method enhances Large Reasoning Model training by reconciling SFT and RL

Researchers have introduced GIFT (Gibbs Initialization with Finite Temperature), a novel method to improve the post-training process for Large Reasoning Models (LRMs). This technique addresses the optimization mismatch between Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) by making SFT targets structurally compatible with the subsequent RL stage. By employing a token-level variational surrogate derived from the Gibbs optimum of KL-regularized RL, GIFT preserves structural diversity through a finite temperature, unlike the distributional collapse seen in standard SFT. Experiments indicate that GIFT outperforms traditional SFT and other baselines when used for RL initialization. AI

IMPACT This new initialization technique could lead to more capable and diverse Large Reasoning Models by improving the synergy between supervised and reinforcement learning phases.

RANK_REASON The cluster contains a research paper detailing a new method for training Large Reasoning Models. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• Gibbs Initialization with Finite Temperature
• Gibbs optimum
• GIFT
• KL-regularized RL
• Large Reasoning Models
• reinforcement learning
• supervised fine-tuning
• Zhengyang Zhao

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →