VeriGate: Verifier-Gated Step-Level Supervision for GRPO
Researchers have developed VeriGate, an extension of Group Relative Policy Optimization (GRPO) designed to improve the training of reasoning models. VeriGate addresses sparse supervision by using process supervision when verifier rewards are degenerate and converts step scores into future-cumulated rewards for better credit assignment. This method has shown significant improvements, increasing average accuracy by up to 20% on the MATH dataset with Qwen2.5-Instruct models and reducing issues like zero-gradient failures and reward hacking. AI
IMPACT Enhances AI reasoning capabilities and training efficiency, potentially leading to more robust and accurate AI systems in complex tasks.