New method tackles bias in LLM reinforcement learning

A new paper introduces History-Aware Adaptive Difficulty Weighting (HA-DW) to address biases in Reinforcement Learning from Verifier Rewards (RLVR) methods, specifically those used in group-based approaches like GRPO. The research identifies that current methods can under-weight correct answers to difficult prompts and insufficiently penalize incorrect answers to easy prompts. HA-DW aims to correct these biased advantage estimations by adapting reweighting schemes based on an evolving difficulty anchor and training dynamics. Experiments on mathematical reasoning benchmarks show that HA-DW improves performance when integrated with GRPO and its variants, suggesting its critical role in more robust RLVR training. AI

IMPACT Improves robustness and efficiency in training large language models for reasoning tasks.

RANK_REASON The cluster contains a research paper detailing a new method for improving LLM training. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• alphaXiv
• arXiv
• CatalyzeX
• DagsHub
• Fengkai Yang
• Gotit.pub
• GRPO
• HA-DW
• History-Aware Adaptive Difficulty Weighting
• Hugging Face
• IArxiv
• Reinforcement Learning from Verifier Rewards
• ScienceCast

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →