Two is better than one: A Collapse-free Multi-Reward RLIF Training Framework
Researchers have developed a new framework for training large language models using Reinforcement Learning from Internal Feedback (RLIF). This multi-reward approach decomposes the training signal into an answer-level reward from cluster voting and a completion-level reward based on token self-certainty. The method incorporates GDPO-based normalization and KL-Cov regularization to enhance stability and prevent collapse, achieving performance close to supervised methods without external ground-truth supervision. AI
IMPACT This new RLIF framework offers a more stable and robust unsupervised training method for LLMs, potentially improving their reasoning capabilities without relying on external human supervision.