GDPval
PulseAugur coverage of GDPval — every cluster mentioning GDPval across labs, papers, and developer communities, ranked by signal.
2 day(s) with sentiment data
-
Human judgment key to AI future, argues Thesis: 2027 keynote
Dan Shipper, in a keynote at the Thesis: 2027 conference, argued that the future of human work in an AI-driven world hinges on human judgment rather than raw AI capabilities. He posited that while aggregate data suggest…
-
New GDPx benchmark tests frontier AI long-horizon capabilities
A new benchmark called GDPx has been introduced to evaluate the long-horizon capabilities of frontier AI models. This successor to GDPval aims to test models on tasks requiring extended planning and execution. Early res…
-
New taxonomy identifies rubric failures in AI evaluation
A new paper introduces the Rubric-Failure Taxonomy (RIFT), a system for identifying and categorizing nine distinct ways rubrics can fail in evaluating AI performance. Researchers demonstrated that RIFT can accurately de…
-
Language models show stable preferences for 'leisure' and aversion to tedium
A new paper published on arXiv details experiments testing the revealed preferences of 20 language models. The study found that models exhibit stable dispositions towards certain tasks, demonstrating tedium aversion, a …
-
New GDPevo benchmark measures AI agent self-evolution
A new benchmark called GDPevo has been developed to measure the self-evolutionary capabilities of AI agents. This benchmark addresses the challenge of evaluating how effectively agents learn from experience and improve …
-
New framework aims to improve AI benchmarks for knowledge work
A new paper proposes a three-step framework for designing and reporting benchmarks for AI systems intended for knowledge work. The approach emphasizes clearly defining the work activity, specifying the testing environme…