Anthropic's Claude Fable 5.2 has reportedly outperformed Google's Gemini 4 Carbon on a Google-developed benchmark, achieving a score of 73.3% compared to Gemini's 71.9%. This performance was observed across 200 visual office tasks. However, the benchmark's results are questioned, with Google's own report suggesting the performance gap is negligible and raising concerns about the validity of vendor-created tests for guiding AI agent purchases. AI
IMPACT Raises questions about the reliability of vendor-created benchmarks for evaluating AI agent capabilities.
RANK_REASON Comparison of AI model performance on a benchmark.
Read on Mastodon — mastodon.social →
AI-generated summary · Google Gemini · from 2 sources. How we write summaries →