SFT data scale and source crucial for LLM generalization, study finds

A new research paper challenges the common understanding that Reinforcement Learning (RL) leads to better generalization in large language models compared to Supervised Fine-Tuning (SFT). The study, focusing on Qwen2.5 and Qwen3 models, demonstrates that carefully curated and scaled SFT datasets can achieve comparable or even superior generalization capabilities to RL. The research highlights that data source and scale are critical factors, with mixed SFT datasets outperforming those generated solely by larger models. By analyzing training dynamics and covering more diverse reasoning signals, SFT can effectively match RL's generalization performance. AI

IMPACT Challenges conventional wisdom on LLM training, suggesting SFT can match RL generalization with optimized data.

RANK_REASON Academic paper published on arXiv detailing new findings about LLM training methods. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• arXiv
• Qwen2.5
• Qwen3
• reinforcement learning
• supervised fine-tuning

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →