Iterative GRPO 使得多轮强化学习训练无需用户模拟器

作者 PulseAugur 编辑部 · [1 个来源] · 2026-09-02 04:00

研究人员推出 Iterative GRPO，这是一种用于训练多轮强化学习（RL）代理的新颖方法，特别适用于对话场景下运行的大型语言模型（LLMs）。该方法将经典的近似策略迭代适应于批处理在线学习范式，其中数据以批次形式收集，然后用于再训练。Iterative GRPO 使用带有学习到的回合级 Q 函数的单轮 RLHF 技术来估计预期的下游回报，从而避免了在训练期间需要复杂的用户模拟器或交互式环境。该方法在多轮谈判场景中已证明了其有效性。 AI

影响通过消除对复杂用户模拟器的需求，实现了更高效的对话式人工智能代理训练。

排序理由该集群包含一篇详细介绍多轮强化学习新方法的学术论文。[lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.LG 阅读 →

AI 生成摘要 · Google Gemini · 来自 1 个来源。我们如何撰写摘要 →

本文如何被排名

Signal score: 30 / 100; Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket: Tool; 该集群包含一篇详细介绍多轮强化学习新方法的学术论文。[lever_c_demoted from research: ic=1 ai=1.0]
Source corroboration: Single-source cluster; Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics: paper, model release; Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance: High; Clearly on-topic for AI-industry coverage.
Story freshness: Breaking (< 6h); Fresh story with cross-source coverage still developing. Ranking may shift as more sources report.

完整方法见我们的编辑标准。

报道来源 [1]

arXiv cs.LG TIER_1 English(EN) · Daniel R. Jiang, Ankur Samanta, Yukai Yang, Jalaj Bhandari, R\'emi Munos, Tyler Lu · 2026-09-02 04:00

迭代式GRPO：通过单轮RLHF实现多轮强化学习的批量在线策略迭代

arXiv:2511.21638v2 Announce Type: replace Abstract: Practical LLM agents often operate over multi-turn conversations where success is determined only after the full interaction ends. Most multi-turn RL methods train via on-policy rollouts, but unlike in single-turn RLHF, the poli…

本文如何被排名

报道来源 [1]

迭代式GRPO：通过单轮RLHF实现多轮强化学习的批量在线策略迭代

相关实体

相关话题