A new benchmark called RWE-bench, designed to test AI capabilities on real-world engineering tasks, has been released. The benchmark features 20 tasks across mechanical, electrical, and aerospace engineering, with results automatically graded against specific requirements and physics simulations. GPT-6 Astra currently leads the benchmark, achieving 37% task completion, followed by Opus 5.5 at 29% and GPT-6.1 Sol at 25%. While these scores may seem low, they significantly outperform human performance on similar tasks, especially for individuals outside of specialized engineering fields. AI
IMPACT This benchmark provides a new metric for AI capabilities in complex, real-world engineering domains, potentially accelerating AI development in these fields.
RANK_REASON New benchmark released for AI on engineering tasks. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
