PlexRL: Cluster-Level Orchestration of Serviceized LLM Execution for RLVR
Researchers have developed PlexRL, a cluster-level runtime designed to improve the efficiency of training large language models (LLMs) for reinforcement learning with verifiable rewards (RLVR). RLVR training is often inefficient due to idle time caused by long-tailed rollouts and tool-induced stalls. PlexRL addresses this by multiplexing LLM services across multiple RLVR jobs, filling idle periods by time-slicing model execution without costly migrations. Evaluations show PlexRL can reduce GPU hour costs by up to 37.58% while maintaining algorithmic flexibility and adding minimal overhead. AI
IMPACT Optimizes LLM training infrastructure, potentially lowering costs and increasing throughput for RLVR applications.