PulseAugur
EN
LIVE 19:07:42

LLM Benchmark Results: Molmo, Qwen3, Hermes, and Mistral Performance Revealed

By PulseAugur Editorial · [7 sources] ·

Independent benchmarks reveal varying performance across several large language models. Molmo 7B-D shows low scores on GPQA and MMLU-Pro, while Qwen3 Omni 30B A3B and Hermes 4 - Llama-3.1 405B demonstrate significantly higher capabilities in reasoning tasks. Mistral Medium 3 also shows strong performance, particularly in long-context reasoning. AI

IMPACT Provides comparative performance data for various LLMs across key benchmarks, aiding in model selection for specific tasks.

RANK_REASON Independent benchmark results for multiple LLMs are reported.

Read on Mastodon — fosstodon.org →

AI-generated summary · Google Gemini · from 7 sources. How we write summaries →

LLM Benchmark Results: Molmo, Qwen3, Hermes, and Mistral Performance Revealed

How we ranked this

Signal score
0 / 100
Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket
Research
Independent benchmark results for multiple LLMs are reported.
Source corroboration
7 independent sources
Strong cross-source corroboration — multiple independent publishers covered this within the clustering window.
Topics
model release, other
Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance
High
Clearly on-topic for AI-industry coverage.
Story freshness
48 days old
Aged out of breaking-news scoring windows; ranking reflects the durable signal from the full source set.
Coverage growth since scoring
+3 source(s) since last score
New sources have picked up this story since our last re-score. Score will update on the next scoring pass.

Full methodology in our editorial standards.

COVERAGE [7]

  1. Mastodon — fosstodon.org TIER_1 English(EN) · [email protected] ·

    📊 Molmo 7B-D — the actual numbers GPQA: 24% MMLU-Pro: 37.1% Humanity's Last Exam: 5.1% Long Context Reasoning: 0% Measured independently, not self-reported → ht

    📊 Molmo 7B-D — the actual numbers GPQA: 24% MMLU-Pro: 37.1% Humanity's Last Exam: 5.1% Long Context Reasoning: 0% Measured independently, not self-reported → https:// olud.ai/leaderboard.html # LLM # Benchmarks # OpenSource # AI

    LINKS olud.ai/leaderboard.html
  2. Mastodon — mastodon.social TIER_1 English(EN) · opensourceaitech ·

    📊 Devstral Small (May '25) — the actual numbers GPQA: 43.4% MMLU-Pro: 63.2% Humanity's Last Exam: 4% Long Context Reasoning: 32% Measured independently, not sel

    📊 Devstral Small (May '25) — the actual numbers GPQA: 43.4% MMLU-Pro: 63.2% Humanity's Last Exam: 4% Long Context Reasoning: 32% Measured independently, not self-reported → https:// olud.ai/leaderboard.html # LLM # Benchmarks # OpenSource # AI

    LINKS olud.ai/leaderboard.html
  3. Mastodon — mastodon.social TIER_1 English(EN) · opensourceaitech ·

    📊 Qwen3.5 9B (Reasoning) — the actual numbers GPQA: 80.6% Humanity's Last Exam: 14.9% Long Context Reasoning: 70% IFBench: 66.7% ⚡ 84.7 tokens/sec 💰 98 intellig

    📊 Qwen3.5 9B (Reasoning) — the actual numbers GPQA: 80.6% Humanity's Last Exam: 14.9% Long Context Reasoning: 70% IFBench: 66.7% ⚡ 84.7 tokens/sec 💰 98 intelligence points per dollar Measured independently, not self-reported → https:// olud.ai/leaderboard.html # LLM # Benchmarks …

    LINKS olud.ai/leaderboard.html
  4. Mastodon — mastodon.social TIER_1 English(EN) · opensourceaitech ·

    📊 GLM-4.5V (Reasoning) — the actual numbers GPQA: 68.4% MMLU-Pro: 78.8% Humanity's Last Exam: 6.3% Long Context Reasoning: 0% 💰 10 intelligence points per dolla

    📊 GLM-4.5V (Reasoning) — the actual numbers GPQA: 68.4% MMLU-Pro: 78.8% Humanity's Last Exam: 6.3% Long Context Reasoning: 0% 💰 10 intelligence points per dollar Measured independently, not self-reported → https:// olud.ai/leaderboard.html # LLM # Benchmarks # OpenSource # AI

    LINKS olud.ai/leaderboard.html
  5. Mastodon — mastodon.social TIER_1 English(EN) · opensourceaitech ·

    📊 Qwen3 Omni 30B A3B (Reasoning) — the actual numbers GPQA: 72.6% MMLU-Pro: 79.2% Humanity's Last Exam: 7.5% Long Context Reasoning: 0% ⚡ 110 tokens/sec 💰 22.1

    📊 Qwen3 Omni 30B A3B (Reasoning) — the actual numbers GPQA: 72.6% MMLU-Pro: 79.2% Humanity's Last Exam: 7.5% Long Context Reasoning: 0% ⚡ 110 tokens/sec 💰 22.1 intelligence points per dollar Measured independently, not self-reported → https:// olud.ai/leaderboard.html # LLM # Ben…

    LINKS olud.ai/leaderboard.html
  6. Mastodon — mastodon.social TIER_1 English(EN) · opensourceaitech ·

    📊 Hermes 4 - Llama-3.1 405B (Reasoning) — the actual numbers GPQA: 72.7% MMLU-Pro: 82.9% Humanity's Last Exam: 10.9% Long Context Reasoning: 22.7% ⚡ 34.9 tokens

    📊 Hermes 4 - Llama-3.1 405B (Reasoning) — the actual numbers GPQA: 72.7% MMLU-Pro: 82.9% Humanity's Last Exam: 10.9% Long Context Reasoning: 22.7% ⚡ 34.9 tokens/sec 💰 5.9 intelligence points per dollar Measured independently, not self-reported → https:// olud.ai/leaderboard.html …

    LINKS olud.ai/leaderboard.html
  7. Mastodon — mastodon.social TIER_1 English(EN) · opensourceaitech ·

    📊 Mistral Medium 3 — the actual numbers GPQA: 57.8% MMLU-Pro: 76% Humanity's Last Exam: 4.1% Long Context Reasoning: 31.7% 💰 15.6 intelligence points per dollar

    📊 Mistral Medium 3 — the actual numbers GPQA: 57.8% MMLU-Pro: 76% Humanity's Last Exam: 4.1% Long Context Reasoning: 31.7% 💰 15.6 intelligence points per dollar Measured independently, not self-reported → https:// olud.ai/leaderboard.html # LLM # Benchmarks # OpenSource # AI

    LINKS olud.ai/leaderboard.html

RELATED ENTITIES

RELATED TOPICS