pass@k
PulseAugur coverage of pass@k — every cluster mentioning pass@k across labs, papers, and developer communities, ranked by signal.
2 day(s) with sentiment data
-
New research identifies pass@k inversion in RLVR, proposes mitigation strategy
A new research paper explores the phenomenon of "pass@k inversion" in reinforcement learning with verifiable rewards (RLVR). This occurs when RLVR improves a model's one-sample accuracy but degrades its performance on t…
-
New research tackles AI code generation evaluation and testing
Two new research papers explore advancements in evaluating AI-generated code. The first, TENET, introduces a framework for repository-level code generation using test-driven development, achieving high Pass@1 scores on …
-
New research questions LLM fine-tuning effectiveness for Dart code decompilation
A new research paper explores the effectiveness of fine-tuning large language models for the neural decompilation of Dart Ahead-of-Time (AOT) binaries. The study found that fine-tuning did not significantly improve pass…
-
New SR-PPO method improves RL for language models with single rollout
Researchers have developed a new method called Single-Rollout Proximal Policy Optimization (SR-PPO) to address the challenges of estimating token-level advantages in reinforcement learning for language models. This appr…
-
New research frames RLVR diversity collapse as overtraining
A new research paper published on arXiv explores the phenomenon of "diversity collapse" in Reinforcement Learning with Verifiable Rewards (RLVR), a technique used to enhance large language models' reasoning. The paper f…
-
New RLVR method tackles code generation redundancy
Researchers have developed a new method called Redundancy-Aware RLVR to improve code generation from large language models. This approach addresses the issue of generated code samples being too similar to each other, wh…