Researchers have developed CurioSFT, a novel supervised fine-tuning method for large reasoning models that aims to preserve exploration capabilities. Unlike standard SFT, which can lead to overconfidence and reduced diversity, CurioSFT uses self-exploratory distillation and adaptive temperature selection to encourage exploration without knowledge forgetting. Experiments show CurioSFT improves performance on in-distribution and out-of-distribution tasks during the SFT phase, and these preserved exploration capabilities translate to significant gains in subsequent reinforcement learning stages. AI
IMPACT This method could lead to more capable and versatile large reasoning models by improving their exploration capabilities during training.
RANK_REASON The cluster contains a research paper detailing a new method for fine-tuning large language models. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
