Process rewards boost small LLM math reasoning accuracy by 10%

A new research paper explores the impact of reward granularity in Reinforcement Learning with Verifiable Rewards (RLVR) for small language models performing mathematical reasoning. The study found that process-level supervision, which rewards intermediate steps, significantly outperformed outcome-only rewards, achieving a nearly 10-percentage point increase in test accuracy on the GSM8K benchmark. Hybrid reward structures generally favored process supervision, though a configuration with low process weight showed a notable anomaly by underperforming pure outcome supervision. Error analysis indicated that process-based models produced more structurally consistent reasoning traces, while outcome-based models were more concise but prone to derivation errors. AI

IMPACT Process-level rewards enhance small language model capabilities in mathematical reasoning, potentially improving their reliability and accuracy in complex tasks.

RANK_REASON Academic paper detailing a novel approach to improving LLM reasoning. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• Anagha Radhakrishna Palandye
• GPT-4o
• GSM8K
• Qwen2.5-0.5B
• RLVR

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →