New Pair-GRPO algorithms enhance LLM alignment stability and generalization

Researchers have introduced the Pair-GRPO family, a novel theoretical framework designed to enhance the stability and generality of reinforcement learning for aligning large language models. This family includes two variants, Soft-Pair-GRPO and Hard-Pair-GRPO, which address limitations in current pairwise preference learning methods by refining reward signals and introducing explicit policy constraints. Experiments on standard LLM alignment benchmarks and a continuous control task show that Pair-GRPO consistently outperforms existing approaches in alignment quality and training stability. AI

影响 Introduces a more stable and generalizable method for aligning LLMs, potentially improving the reliability of AI systems.

排序理由 This is a research paper detailing a new theoretical framework and experimental results for improving LLM alignment. [lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.LG 阅读 →

• HalfCheetah-v4
• Hard-Pair-GRPO
• HH-RLHF
• LLM
• Pair-GRPO
• RLHF
• Soft-Pair-GRPO
• UltraFeedback
• GRPO

• 论文
• 安全

AI 生成摘要 · Google Gemini · 来自 1 个来源。 我们如何撰写摘要 →