NgramRL accelerates LLM training with rollout-aware n-gram speculative decoding

Researchers have developed NgramRL, a novel method to accelerate reinforcement learning (RL) post-training for large language models. This technique addresses the significant cost of autoregressive rollout generation, which can account for over 70% of training time. NgramRL utilizes a per-prompt n-gram drafter constructed from recent rollout history, eliminating the need for auxiliary models or persistent GPU state. An adaptive fan-out prefetcher further optimizes performance by predicting verification outcomes, leading to substantial improvements in token commitment per forward pass. AI

IMPACT NgramRL could significantly reduce the computational cost of training large language models, potentially making advanced RL post-training more accessible.

RANK_REASON The cluster contains a research paper detailing a new method for accelerating LLM training. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• large-language models
• NgramRL
• Nilesh Prasad Pandey
• Qwen3_8B
• reinforcement learning
• speculative decoding
• SuffixDecoding

• paper
• infra

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →