Humanity's Last Exam
PulseAugur coverage of Humanity's Last Exam — every cluster mentioning Humanity's Last Exam across labs, papers, and developer communities, ranked by signal.
- instance of Deep Research 90%
- instance of Claude Opus-5 90%
- instance of MMLU-Pro 70%
- instance of long-context reasoning 70%
- instance of SciCode 70%
- competes with Claude Fable-5 70%
- competes with Claude Opus-5 70%
- instance of GPQA Diamond 70%
- competes with GPT 5.6 "Sol" 70%
- instance of BrowseComp+ 70%
- instance of DeepSearchQA 70%
- instance of ARC-AGI-2: A New Challenge for Frontier AI Reasoning Systems 70%
10 day(s) with sentiment data
-
DeepSeek releases free coding model, undercuts search rivals
DeepSeek has released an updated version of its open-source coding model, V4.1-Flash-1017, which reportedly outperforms Claude Opus-5 and GPT-5.6 Sol on a specific coding benchmark. The company also launched free-to-acc…
-
Molmo 7B-D benchmark results show mixed performance
The Molmo 7B-D language model has been evaluated on several benchmarks, showing specific scores on GPQA, MMLU-Pro, and Humanity's Last Exam. The model also scored 0% on long-context reasoning tasks. These results were m…
-
Eigen-Agent framework boosts LLM scientific reasoning with novel retrieval and collaboration
Researchers have developed Eigen-Agent, a novel framework designed to enhance scientific reasoning in large language models. This system integrates knowledge with minimal disruption through a Monitor-based retrieval mod…
-
Exaone 4.0 benchmarks show mixed results, scoring low on reasoning
Exaone 4.0, a 1.2 billion parameter model, has demonstrated specific performance metrics on various benchmarks. Independently measured results show it achieved 42.4% on GPQA, 50% on MMLU-Pro, and 5.7% on Humanity's Last…
-
Gemini 4 Argon, Claude Opus 5.5, and GPT-5.3 Codex Lead Latest AI Model Comparisons
In February 2026, AI labs released several advanced models, including Gemini 3.1 Pro, Claude Opus 4.6, and GPT-5.3 Codex, each demonstrating strengths in different areas of software development. Gemini 3.1 Pro excelled …
-
K2-V2 (Medium) benchmark scores reveal reasoning gap
Independent evaluations of the K2-V2 (Medium) model have revealed its performance on various benchmarks. While it achieved scores of 59.8% on GPQA and 76.1% on MMLU-Pro, its performance on Humanity's Last Exam dropped t…
-
LLM evaluation method adapts psychometric theory for fairer benchmark comparisons
Researchers have developed a new method for evaluating large language models (LLMs) by adapting Item Response Theory (IRT) from psychometrics. This approach treats each benchmark as an "item" and models the LLM's "abili…
-
Anthropic's Claude Haiku 5.5 boosts operational skills, introduces adaptive reasoning
Anthropic has released Claude Haiku 5.5, a significant upgrade featuring enhanced operational capabilities and adaptive reasoning. The new model shows substantial improvements in computer operations, knowledge work, and…
-
OpenAI launches GPT-6 with interactive UI; Anthropic slashes Haiku costs
OpenAI has released GPT-6, featuring an "Intelligent UI" that integrates interactive elements directly into the chat interface, transforming it into an application generator. Concurrently, Anthropic has significantly re…
-
Open-source LLMs evaluated: DeepSeek V4 leads benchmarks
Several open-source large language models have been independently evaluated on various benchmarks, with DeepSeek V4 Flash 0420 achieving top scores in GPQA (89.4%) and Humanity's Last Exam (34.8%). Qwen3.5 4B also showe…
-
Hugging Face leaderboards: How benchmarks are populated and viewed
Hugging Face maintains leaderboards for 48 official benchmarks, which are populated by model authors submitting results via .eval_results YAML files in their model repositories. These leaderboards are automatically asse…
-
DeepSeek V4 Pro 0813 benchmarks show 10.6% on Humanity's Last Exam
DeepSeek V4 Pro 0813 has demonstrated specific performance metrics on benchmarks including Humanity's Last Exam, Long Context Reasoning, and SciCode. The model achieved 10.6% on Humanity's Last Exam, 50.7% on Long Conte…
-
DeepSeek V4 Pro and Nova 2.0 Lite benchmarks revealed · 2 sources tracked
Independent benchmarks reveal the performance of two large language models, DeepSeek V4 Pro and Nova 2.0 Lite. DeepSeek V4 Pro achieved high scores in reasoning-focused tasks, with 92.8% on GPQA and 80.3% on Long Contex…
-
GPT-6 Astra and Claude Fable 5.1 face off in benchmark showdown
A direct comparison between OpenAI's GPT-6 Astra and Anthropic's Claude Fable 5.1 reveals distinct strengths for each advanced AI model. GPT-6 Astra excels in handling massive codebases with its 1.1M context window and …
-
MiniMax M1 40k struggles on real-world reasoning benchmark
An independent benchmark evaluation of the MiniMax M1 40k model revealed a significant performance decrease, scoring only 7.8% on the Humanity's Last Exam. This result highlights the ongoing challenges in achieving robu…
-
Google and Meta release new coding-focused AI models
Google and Meta have both released new coding-focused AI models, Gemini 3.8 Flash and Muse Spark 1.3, respectively. These models are designed for high-volume coding and agentic tasks at a lower price point than flagship…
-
AI oversight monitors improve with answer access, but reasoning verification remains weak
A new research paper titled "The Answer Is Not the Argument" explores the effectiveness of chain-of-thought monitoring for AI oversight. The study found that providing AI monitors with a trusted reference answer signifi…
-
Anthropic launches Claude Fable 5.1 with improved coding, lower costs · 10 sources tracked
Anthropic has released its latest AI models, Claude Fable 5.1 and Mythos 5.1, which offer improved performance in coding and scientific research. Fable 5.1, the generally available version, shows significant gains on be…
-
Anthropic's Claude Fable 5.1 released with benchmark gains, cost debates · 10 sources tracked
Anthropic has released Claude Fable 5.1, a new iteration of its AI model, which is now available on platforms like Cursor and Perplexity. This release boasts significant improvements in benchmarks, particularly in agent…
-
LLM Benchmark Results: Molmo, Qwen3, Hermes, and Mistral Performance Revealed
Independent benchmarks reveal varying performance across several large language models. Molmo 7B-D shows low scores on GPQA and MMLU-Pro, while Qwen3 Omni 30B A3B and Hermes 4 - Llama-3.1 405B demonstrate significantly …