SWE-rebench leaderboard adds 110 new Python tasks for AI models

The SWE-rebench leaderboard has been updated with 110 new Python tasks from GitHub PRs spanning March, April, and May. This update focuses on evaluating models' ability to read real issues, edit code, and pass test suites. Future updates will include more models like Gemini Flash 3.5 and DeepSeek v4 Pro, alongside multilingual tasks and options for local development. AI

IMPACT Provides updated benchmarks for AI models on software engineering tasks, influencing future development and evaluation strategies.

RANK_REASON The cluster reports on a benchmark leaderboard update for AI models, which is a form of research evaluation. [lever_c_demoted from research: ic=1 ai=1.0]

Read on r/LocalLLaMA →

• Cursor (Composer 2.5)
• DeepSeek v4 Pro
• Gemini Flash 3.5
• GPT-5.5
• Kimi K2.6
• Opus 4.7
• Qwen3.5-397B-A17B
• SWE-rebench

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →