A new research paper explores the phenomenon of "pass@k inversion" in reinforcement learning with verifiable rewards (RLVR). This occurs when RLVR improves a model's one-sample accuracy but degrades its performance on tasks requiring multiple attempts (higher k). The paper argues this is an absence-of-evidence failure, where rare correct trajectories are lost during RLVR training. A proposed method, Per-Problem Base Anchoring (PBA), aims to mitigate this by sharpening prompts and anchoring risky ones to the base model's distribution, showing improved results on benchmarks like Omni-MATH-Test. AI
IMPACT This research could lead to more robust AI models by addressing a specific failure mode in verifiable reward systems, potentially improving performance in complex reasoning tasks.
RANK_REASON Academic paper detailing a novel research finding and proposed method. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
