New CAST method improves LLM reasoning via self-distillation

Researchers have developed CAST, a novel self-distillation method designed to enhance reinforcement learning with verifiable rewards (RLVR) in large language models, particularly for Group Relative Policy Optimization (GRPO). CAST addresses the limitations of sparse outcome-level rewards and the potential misalignment of token-level guidance from On-Policy Self-Distillation (OPSD). By using an answer-free self-teacher and incorporating bidirectional local advantage sign reversal, CAST aims to provide more effective token-level feedback aligned with trajectory correctness, as demonstrated in experiments on mathematical reasoning tasks. AI

IMPACT Introduces a new technique to improve the training of LLMs for complex reasoning tasks.

RANK_REASON This is a research paper describing a new method for improving LLM reasoning. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• CAST
• GRPO
• RLVR

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →