Researchers have discovered that specific starting tokens in a base model's response can significantly improve its reasoning capabilities, bringing its performance close to that of models trained with reinforcement learning (RL). By manipulating these token cues, such as using "\n\nOkay" or "Alright,", performance on math benchmarks like MATH-500 can increase dramatically. The study further reveals that these cues are learned from the training data, and RL training makes them more probable. Causal interventions on the training data can even turn arbitrary words into effective reasoning cues, demonstrating a direct link between data content and model behavior, including safety-related responses. AI
IMPACT Suggests that fine-tuning base models with specific data cues could significantly boost reasoning abilities without extensive RL training.
RANK_REASON Academic paper detailing a new finding about model behavior. [lever_c_demoted from research: ic=1 ai=1.0]
Read on Hugging Face Daily Papers →
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
