ProgramBench
PulseAugur coverage of ProgramBench — every cluster mentioning ProgramBench across labs, papers, and developer communities, ranked by signal.
1 day(s) with sentiment data
-
MindForge pipeline trains small LLMs for full-cycle software engineering
Researchers have developed MindForge, a novel pipeline designed to train smaller language models in comprehensive software engineering tasks. This system converts open-source command-line programs into source-free train…
-
Fable 5 benchmark shows double Opus 4.8 performance
A new benchmark, ProgramBench, has been used to evaluate Fable 5, with results suggesting it significantly outperforms Opus 4.8. The benchmark creator noted that Fable 5's performance was double that of Opus 4.8, even w…
-
ProgramBench coding benchmark fails frontier models due to impossible undocumented tests
A new coding benchmark called ProgramBench, designed to evaluate frontier AI models, has been criticized for being potentially impossible to solve. The benchmark requires models to reimplement programs based on limited …
-
ProgramBench benchmark finds language models struggle to build software from scratch
Researchers have introduced ProgramBench, a new benchmark designed to evaluate the holistic software development capabilities of language models. The benchmark challenges AI agents to architect and implement entire code…