New CERO method optimizes LLM reinforcement learning with adaptive rollout allocation

Researchers have developed a new method called CERO for optimizing reinforcement learning post-training in large language models. CERO adaptively allocates a fixed budget of rollouts across different prompts, unlike previous methods that used a static budget. This approach uses Bayesian estimates of prompt success probabilities to determine the value of additional rollouts, leading to improved sample efficiency. Experiments showed CERO outperformed existing methods on mathematical reasoning tasks with various open-weight LLMs. AI

IMPACT Improves sample efficiency in LLM training, potentially leading to faster development of more capable models.

RANK_REASON Academic paper detailing a new method for LLM post-training. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• CERO
• LLM
• reinforcement learning

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →