Rethinking Reward Supervision: Rubric-Conditioned Self-Distillation
Researchers have introduced Rubric-Conditioned Self-Distillation, a novel framework for post-training reasoning language models. This method utilizes structured, fine-grained feedback from rubrics to guide self-distillation, offering more detailed credit assignment than traditional scalar reward signals. The framework involves a two-stage pipeline that first generates task-specific rubrics and then trains a rubric-guided reasoner. Evaluations on science reasoning benchmarks demonstrate that this approach effectively translates rubric criteria into token-level guidance, outperforming existing methods like GRPO and OPSD. AI
IMPACT This framework could lead to more capable reasoning language models by providing more nuanced feedback during training.