AI models tested on complex benchmark; DeepSeek 4 Pro servers melt

A user is attempting to benchmark the DeepSeek 4 Pro model, but its servers are experiencing high load. The benchmark involves a complex reverse-engineering task to create a tool for building Apollo GraphQL hashes. So far, no open-weight models have successfully completed the benchmark, while proprietary models like Anthropic's Opus 4.7 and OpenAI's GPT 5.5 have demonstrated success. AI

IMPACT Provides comparative performance data for proprietary models on a complex reverse-engineering task.

RANK_REASON User is running a benchmark on a model and comparing results, which falls under research.

Read on Mastodon — fosstodon.org →

• Anthropic Opus 4.6
• Anthropic Opus 4.7
• Apollo GraphQL
• DeepSeek 4 Pro
• Gemini Pro 3.1
• GitHub Copilot
• Kimi K2.6
• Ollama
• OpenAI GPT-5.4
• OpenAI GPT 5.5
• OpenCode Go
• OpenRouter

• model release
• paper

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →