The StarSkirmish benchmark, designed to test long-horizon reasoning and agentic coding in LLMs, has released its v0.2 results. In this benchmark, models are tasked with writing a StarCraft: Brood War bot in C++ within one hour, with their performance evaluated against other LLM-generated bots and human-written bots. GPT-6 Astra emerged as the top performer, demonstrating a significant lead over other models, while Gemini 4 Argon, Claude Opus 5.5, and Claude Fable 5.1 showed comparable, albeit lower, scores. The benchmark also highlighted that even the best-performing models are still considerably weaker than top human-written bots, indicating room for future improvement. AI
IMPACT Sets a new benchmark for agentic coding and long-horizon reasoning in LLMs, pushing the frontier of AI capabilities in complex strategic games.
RANK_REASON LLM benchmark results published with detailed methodology and comparative analysis. [lever_c_demoted from research: ic=1 ai=1.0]
Read on HN — anthropic stories →
- Anthropic
- Claude Fable 5.1
- Claude Opus 5.5
- Gemini 4 Argon
- GPT-6.1 Sol
- GPT-6 Astra
- Meta*
- Muse Wonder Full
- OpenAI
- StarSkirmish
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
