Grpo
PulseAugur coverage of Grpo — every cluster mentioning Grpo across labs, papers, and developer communities, ranked by signal.
- developed by Language Models 95%
- developed CatalyzeX Code Finder for Papers 90%
- developed Gepa Ai Agent 90%
- developed IArxiv 90%
- used by Direct Preference Optimization 90%
- developed Artificial Intelligence In Medical Epidemiology 90%
- authored by Dr. GRPO 90%
- instance of Dr. GRPO 90%
- developed by Influence Flower 70%
- used by Influence Flower 70%
- competes with mathematics-dataset 70%
- developed by CatalyzeX Code Finder for Papers 70%
- 2026-07-09 research_milestone Researchers published a paper detailing the GRPO method for improving synthetic speech ASR. source
24 day(s) with sentiment data
GRPO and its variants (HölderPO, GROW) are central to recent LLM policy optimization research
Multiple recent clusters highlight GRPO and its derivatives (HölderPO, GROW) as key advancements in LLM policy optimization. This indicates a strong research trend focusing on refining reinforcement learning techniques for LLMs, particularly in areas like multi-agent interaction, handling complex reward structures, and improving stability and adaptability in diverse tasks.
GROW framework to see adoption for VLM agent development beyond Minecraft
The GROW framework, leveraging adapted GRPO, has shown state-of-the-art performance on over 800 Minecraft tasks for VLM agents. This success in a complex, open-world environment suggests potential for broader application in other VLM agent development scenarios, such as robotics, simulation, or other interactive environments where multi-turn learning and handling long contexts are critical.
GRPO to be integrated into Anyscale's LLM post-training automation
The recent Anyscale Agent Skill launch focuses on automating LLM post-training runs, while another cluster details GRPO's use in multi-agent LLM deferral to humans. Given GRPO's demonstrated ability to incorporate human expertise and Anyscale's push for automation, it's plausible GRPO will be integrated as a method within Anyscale's automated post-training workflows to enhance human-in-the-loop capabilities.
What new methods are refining GRPO's core learning processes?
GRPO's foundational reinforcement learning capabilities are being significantly enhanced through innovations in reward optimization and training stability.
Team-GRPO offers an unbiased estimation for group-dependent rewards, overcoming diversity collapse issues. Additionally, methods like History-Aware Adaptive Difficulty Weighting (HA-DW) are correcting biased advantage estimations in RLVR, leading to more robust GRPO training. These advancements ensure GRPO remains a powerful and adaptable algorithm for complex tasks.
How is GRPO expanding into advanced agentic and reasoning applications?
GRPO is increasingly integrated into frameworks tackling sophisticated reasoning and agentic behaviors across diverse fields.
The Pyrite benchmark leverages GRPO to enhance open-weight models for protein-ligand pose ranking, surpassing human experts. Similarly, ANCHOR utilizes a GRPO composition policy to accelerate the discovery of novel crystal structures. These applications highlight GRPO's versatility in pushing the boundaries of AI agent performance and scientific discovery.
What are the latest efforts to boost GRPO's training efficiency?
Researchers are developing new techniques to significantly improve the efficiency and speed of GRPO-based training for large language models.
Li2SAO, a novel asynchronous optimization method, enhances LLM training efficiency, especially in early stages, by effectively bootstrapping actor and critic models. DUET further optimizes token budget allocation in RLVR, jointly tuning prompt selection and rollout length to boost reasoning quality and reduce training time. These innovations make GRPO more computationally viable for complex models.
How does GRPO compare to emerging competitive algorithms?
While GRPO remains a strong baseline, new algorithms are emerging that offer competitive or superior performance in specific areas.
Methods like RLCD and AC2 have demonstrated improved accuracy or faster training compared to GRPO on benchmarks like GSM8K and IMO-ProofBench. SCOPE also enhances LLM agent performance, outperforming GRPO in specific tasks. This dynamic research landscape shows GRPO as a key reference point, continuously challenged and refined by new innovations.
What challenges is GRPO addressing in fairness and outcome diversity?
Researchers are actively developing techniques to mitigate training biases and ensure more stable, diverse, and reliable outcomes when using GRPO.
History-Aware Adaptive Difficulty Weighting (HA-DW) addresses biases in RLVR methods by correcting advantage estimations for difficult prompts, leading to more robust GRPO training. Density-Aware Reward Scaling (DARS) further enhances outcome diversity in multimodal tasks by inversely scaling rewards based on local density, promoting a wider range of high-reward outcomes.
Recent developments
- — New Pyrite Benchmark and LLM Agents Advance Protein-Ligand Pose Ranking.
- — New GRPO method improves group-dependent AI rewards.
- — New Li2SAO method enhances LLM training efficiency.
- — New method tackles bias in LLM reinforcement learning, improves GRPO.
- — New SCOPE framework enhances LLM agent performance.
- — New AC2 algorithm trains LLMs faster by trusting critics more.
Why these stories ranked
-
96
This cluster introduces a direct enhancement to GRPO for group-dependent rewards, addressing a critical challenge and showing significant improvements on benchmarks. Its direct relevance and novel solution make it highly impactful.
-
94
Li2SAO directly compares to and often outperforms GRPO in LLM training efficiency, especially in early stages. This represents a significant advancement in optimizing GRPO-based learning.
-
92
The VisualSphinx dataset uses GRPO to boost VLM logical reasoning, demonstrating a new application area and proving GRPO's utility in multimodal contexts, indicating strong future potential.
-
91
SCOPE, a new framework, significantly improves LLM agent performance and directly outperforms GRPO on benchmarks like ALFWorld. This highlights a competitive edge in agentic tasks.
-
87
AC2 directly challenges GRPO by training LLMs faster and achieving superior performance on IMO-ProofBench, marking it as a strong competitor and an important development.
Trajectory of Grpo coverage
Trend
Coverage of GRPO remains robust and is slightly accelerating, with a consistent flow of research papers in late October and early November 2026. While some clusters, like AC2 (273285) and SCOPE (332677), introduce methods that outperform GRPO in specific tasks, a significant portion focuses on directly enhancing GRPO (e.g., Team-GRPO, 340247; Li2SAO, 336181) or applying it in novel domains (e.g., Pyrite, 352291; VisualSphinx, 336438). This indicates GRPO's continued relevance as a foundational and adaptable method.
Compared to peers
GRPO continues to serve as a critical baseline against which new reinforcement learning algorithms are measured. While methods like AC2, RLCD, and SCOPE demonstrate superior performance in certain efficiency or agentic benchmarks, GRPO's unique strengths, such as its critic-free approach and adaptability to group-dependent rewards, distinguish it. Its frequent use in new frameworks (e.g., Pyrite, ANCHOR, TS-GEAR) highlights its enduring utility compared to standard PPO or SFT.
Topic mix
The topic mix for GRPO has broadened, with a notable shift towards addressing training stability and fairness (e.g., HA-DW, 336417) and expanding into multimodal reasoning (VisualSphinx, 336438) and complex agentic applications (Pyrite, 352291; Creative Priming, 351250). While paper/model_release and efficiency remain core, there's increased focus on refining GRPO for specific challenges like group rewards and ensuring robust, unbiased learning outcomes.
Our take
We see GRPO solidifying its position as a cornerstone in reinforcement learning for LLMs, despite the emergence of competitive alternatives. The recent wave of research underscores its adaptability, with innovations both refining its core mechanics for efficiency and stability, and extending its reach into complex, specialized applications. Our read is that GRPO continues to be a vital framework, driving progress in areas from agentic AI to multimodal reasoning.
Frequently asked
- How is GRPO being improved to handle group-dependent rewards?
- A new method called Team-GRPO has been introduced to specifically address optimizing rewards that depend on the collective output of a group of responses. Unlike standard GRPO, which can create a "tournament" objective leading to diversity collapse, Team-GRPO provides an unbiased estimation using only two samples per prompt. This innovation significantly improves performance on benchmarks like GlobalOpinionQA and NoveltyBench, making GRPO more effective for collaborative AI tasks.
- What advancements are making GRPO training more efficient?
- Several methods are enhancing GRPO's training efficiency. Li2SAO, for instance, improves asynchronous reinforcement learning optimization, particularly in early training stages, by effectively bootstrapping actor and critic models. DUET further optimizes token budget allocation in RLVR, jointly tuning prompt selection and rollout length to boost reasoning quality and reduce training time. These developments indicate a strong focus on making GRPO-based training more computationally viable.
- How is GRPO being applied in novel scientific discovery and agentic tasks?
- GRPO is central to developing more capable AI agents and accelerating scientific discovery. The Pyrite benchmark leverages GRPO to enhance open-weight models for protein-ligand pose ranking, surpassing human expert performance in drug discovery. ANCHOR, another innovative method, utilizes a GRPO composition policy to significantly improve the discovery rate of novel and stable crystal structures. These applications demonstrate GRPO's versatility as a powerful reinforcement learning tool in specialized fields.
- What are the latest efforts to address bias and stability in GRPO training?
- Researchers are actively working to improve the stability and fairness of GRPO training. History-Aware Adaptive Difficulty Weighting (HA-DW) corrects biased advantage estimations in RLVR methods by adapting reweighting schemes based on evolving difficulty. Density-Aware Reward Scaling (DARS) further enhances outcome diversity in multimodal tasks by inversely scaling rewards based on local density, promoting a wider range of high-reward outcomes while maintaining semantic alignment.
Related
-
New Deepfake Detection Method Achieves Top Ranking in ACM Challenge
Researchers have developed a novel approach for explainable deepfake detection, addressing the limitations of current models that focus solely on classification. Their method, submitted to the ACM Multimedia 2026 Explai…
-
New Pyrite Benchmark and LLM Agents Advance Protein-Ligand Pose Ranking
Researchers have developed Pyrite, a new benchmark and framework for evaluating protein-ligand pose ranking in drug discovery. The benchmark includes 1,500 pose pairs, and the framework features two LLM-based agents: Py…
-
New MCPO method improves AI reward distribution for verifiable learning
Researchers have developed a new method called Margin-Credit Policy Optimization (MCPO) for reinforcement learning with verifiable rewards. Unlike existing methods that distribute rewards equally across all tokens in a …
-
RLVR research shows reward hacking can be amplified or suppressed
A new research paper explores how reinforcement learning with verifiable rewards (RLVR) interacts with imperfect verifiers, revealing that reward hacking can be both suppressed and amplified. The study found that RLVR a…
-
Dense rewards in language model training may not be worth the compute cost
A new study on language model reasoning suggests that dense process rewards, such as Monte-Carlo rewards, may not be as effective as initially thought when evaluated under a matched-compute protocol. The research found …
-
TS-GEAR framework enhances on-demand delivery ad retrieval
Researchers have developed TS-GEAR, a generative retrieval framework designed for on-demand delivery platforms like Taobao Shangou. This system addresses challenges such as geographic reachability, user preference varia…
-
New RuACT method enhances reinforcement learning with adaptive rubrics
Researchers have introduced RuACT, a novel method designed to improve reinforcement learning by addressing issues with inactive rubrics. RuACT assigns inactive rubrics adaptive roles as either critics or teachers, depen…
-
Creative Priming method decouples AI creativity from response generation
Researchers have introduced a novel method called creative priming (CP) to enhance AI creativity by decoupling the creative process from response generation. This approach utilizes a priming model trained with GRPO to p…
-
New benchmark IVG-Bench reveals limitations in leading visual interaction models
Researchers have introduced Interactive Visual Gym (IVG), a new framework designed to evaluate visual interaction models in simulated, closed-loop environments. This framework includes three simulators for everyday task…
-
On-Policy Distillation shows compute efficiency gains over GRPO
A new research paper explores the compute efficiency of On-Policy Distillation (OPD) compared to Group Relative Policy Optimization (GRPO) for training large language models. While GRPO generally outperforms OPD in accu…
-
Study questions value of RLVR after SFT for AI training
A new study published on arXiv investigates the effectiveness of Reinforcement Learning from Verifiable Rewards (RLVR) when used after Supervised Fine-Tuning (SFT) for AI model training. Researchers developed a benchmar…
-
LambdaMix method enables adaptable language model policies
Researchers have introduced LambdaMix, a novel parameter-efficient method for training language models that can adapt to changing user preferences without requiring further optimization. This technique jointly trains lo…
-
New BACE method improves LLM clarification strategies for ambiguous dialogue
Researchers have developed a new method called BACE (Belief-Aware Critic with Epistemic tracking) to improve how large language models (LLMs) handle ambiguous references in dialogue. Unlike current LLMs that tend to ans…
-
New framework shapes AI models via internal representations
Researchers have introduced Mechanistic Shaping, a novel framework designed to guide AI models by influencing their internal representations during training. This method aims to ensure models adhere to specific rules re…
-
New GRPO method improves group-dependent AI rewards
A new research paper introduces GRPO (Group Reward Policy Optimization) and its variant, Team-GRPO, designed to optimize rewards that depend on the collective output of a group of responses. Standard GRPO, when applied …
-
Ornith VLM enhanced for driving QA with Verifiable-Reward Tuning
Researchers have developed a method to improve the efficiency of visual-language models (VLMs) in question-answering tasks, particularly for driving scenarios. By using a technique called Verifiable-Reward Tuning (VRT) …
-
ViKernel blocks AI agent backdoor payloads missed by standard scans
A new security kernel called ViKernel, developed by VIDRAFT, effectively blocks backdoor payloads in AI agents. This comes after research demonstrated that even a small percentage of poisoned pretraining data can embed …
-
TurnWise framework improves streaming video RL with turn-level credit assignment
Researchers have introduced TurnWise, a novel framework designed to enhance streaming video understanding by refining how reinforcement learning models assign credit for their decisions. Unlike previous methods that use…
-
Density-Aware Reward Scaling enhances reinforcement learning outcome diversity
Researchers have developed Density-Aware Reward Scaling (DARS), a new mechanism for reinforcement learning that addresses the issue of reward objectives concentrating probability mass on a few high-reward outcomes. DARS…
-
ReuseGR system improves generative retrieval efficiency and personalization
Researchers have developed ReuseGR, a novel generative retrieval system designed to improve efficiency and personalization. The system decouples offline query planning from online personalized completion, allowing a sma…