New Balanced Aggregation method improves GRPO training for LLMs

Researchers have identified and proposed a solution for aggregation bias in GRPO-style training, a method used to enhance reasoning and code generation in large language models. The study reveals that standard GRPO's aggregation methods, sequence and token aggregation, introduce distinct optimization biases. To counter this, they introduce Balanced Aggregation (BA), a drop-in replacement that improves training stability and performance. Experiments with Qwen2.5-Math-7B and Qwen3-1.7B models demonstrated BA's effectiveness across various reasoning and coding benchmarks. AI

IMPACT Introduces a novel aggregation method that enhances training stability and performance for LLMs in reasoning and code generation tasks.

RANK_REASON This is a research paper detailing a new method for improving existing LLM training techniques. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• DAPO-17k
• Polaris
• Qwen2.5-Math-7B
• Qwen3-1.7B
• GRPO

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →