A comparison of various large language models (LLMs) on International Mathematical Olympiad (IMO) 2026 problems revealed that frontier models like Sol and Fable achieved near-perfect scores, regardless of the orchestration harness used. Anthropic's Sonnet and Opus models performed poorly without a harness, but showed improvement with provider-specific harnesses and further gains with AutoFyn, a custom multi-agent harness. Even with AutoFyn, these models did not match the performance of the frontier models. Open-weight model GLM performed similarly to Sonnet without a harness, also improving with AutoFyn. The study noted that hallucination remains an issue, with models sometimes presenting false solutions, and the hardest problems required novel ideas beyond retrieval and verification capabilities. AI
IMPACT Highlights the current capabilities and limitations of LLMs in complex reasoning and problem-solving, indicating frontier models still hold a significant advantage.
RANK_REASON Research paper comparing LLM performance on a novel benchmark. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
