A new benchmark called Science Prediction Universe (SPU) has been released to evaluate AI's ability to predict the outcomes of wet-lab experiments. Developed by UniPat AI with contributions from top universities like Stanford and Harvard, SPU includes 116 tasks across chemistry, materials science, biology, and medicine. Initial evaluations show that even leading models like Opus 5.5 and GPT-6 Astra struggle with this complex task, with average scores often barely exceeding random chance. The benchmark highlights that simply increasing computational resources or reasoning intensity does not guarantee better predictive performance, suggesting a need for more nuanced approaches to AI in scientific research. AI
IMPACT This benchmark may push AI development towards more robust scientific reasoning and experimental prediction capabilities.
RANK_REASON Release of a new benchmark for AI in scientific research. [lever_c_demoted from research: ic=1 ai=1.0]
- Claude
- Cornell University
- Fable 5.1
- GPT-6 Astra
- Harvard University
- Opus 5.5
- Qwen3.8
- Science Prediction Universe
- Stanford University
- UniPat AI
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
