SG-OPD: Sign-Gated On-Policy Distillation via Sign-Consistency Gating and Phased Teacher Sampling
Researchers have developed Sign-Gated On-Policy Distillation (SG-OPD), an advancement in on-policy distillation techniques. This new method incorporates a binary verifier to filter teacher signals, leading to improved performance in mathematical reasoning tasks. SG-OPD addresses limitations in standard on-policy distillation by ensuring better alignment between student and teacher trajectories and more reliable teacher preferences at the token level. Experiments demonstrated significant gains, with SG-OPD outperforming standard on-policy distillation by an average of 1.98% at the per-sample level and 7.50% at the per-question level on mathematical reasoning benchmarks. AI
IMPACT This new distillation method could lead to more capable AI models for complex reasoning tasks like mathematics.