AGPO: Adaptive Group Policy Optimization with Dual Statistical Feedback
Two new research papers introduce methods to improve the training of large language models using reinforcement learning. One paper addresses the issue of "advantage collapse" in Group Relative Policy Optimization (GRPO) by introducing a diagnostic metric and an adaptive extension called AVSPO. The other paper proposes Adaptive Group Policy Optimization (AGPO), which uses group-level statistics to dynamically adjust training parameters like clipping and decoding temperature, outperforming existing methods on several benchmarks. AI
IMPACT These new reinforcement learning techniques aim to enhance LLM reasoning capabilities and training stability, potentially leading to more robust and accurate models.