GPT-6 Astra leads AI coding benchmark with 69.9% check pass rate · 1 source tracked

A new benchmark called Co-Design Bench, detailed in an arXiv paper, evaluated ten AI coding models on real-world editing tasks. GPT-6 Astra emerged as the top performer, successfully completing 26.4% of tasks fully and passing an average of 69.9% of hidden checks. The benchmark's methodology involved providing models with a web app and a user's change request, then assessing their performance against unrevealed acceptance criteria. Researchers noted that the choice of coding harness significantly impacted model scores, emphasizing the need to report both model and harness together. AI

IMPACT Sets a new benchmark for AI coding assistants, highlighting the importance of task completion and the impact of the underlying harness.

RANK_REASON The cluster reports on a new benchmark and its results for AI coding models, based on an arXiv paper. [lever_c_demoted from research: ic=1 ai=1.0]

Read on dev.to — LLM tag →

• Alibaba Group
• arXiv
• Claude Code
• Co-Design Bench
• GPT-6 Astra
• kimi-k3
• Playwright
• Qwen3.8-27B
• The Decoder

• product
• paper

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →