Oracle Performance Gap

RL benchmarks fail to reveal LLM failures, study finds

A new research paper questions the effectiveness of current benchmarks in evaluating reinforcement learning (RL) for large language models (LLMs). The study found that training directly on test sets of existing benchmar…