Brief

A new benchmark called CUSP has been developed to evaluate AI's ability to forecast scientific progress. The study found that current frontier AI models struggle with predicting the realization and timing of scientific advances, despite being able to identify plausible research directions. Performance varies significantly across scientific domains, with AI progress being more predictable than advances in biology, chemistry, and physics, and models exhibit overconfidence in their predictions. AI

Researchers are developing new benchmarks to address the safety risks of AI agents, particularly in multi-agent and interactive environments. GT-HarmBench evaluates frontier models in game-theoretic scenarios, revealing significant failures in high-stakes situations. Boiling the Frog and AgentThreatBench focus on incremental attacks and indirect prompt injections that traditional benchmarks miss, assessing both task utility and security. These efforts aim to create more robust evaluations for AI systems operating beyond simple text generation. AI