研究人员推出了SWE-Bench ProMax，这是一个旨在评估AI编码代理在复杂、大规模代码重构任务上的新基准。该基准解决了现有评估中的局限性，例如测试缺陷以及模型复现训练数据的能力。SWE-Bench ProMax包含170个专家精心策划的实例，涵盖七种编程语言，每个实例平均修改11.4个文件和261.6行代码。初步实验表明，即使是先进的AI模型，解决率也仅为41.2%，这表明SWE-Bench ProMax对AI编码能力提出了重大且目前具有挑战性的测试。 AI
影响 该基准将通过提供对AI处理复杂、多文件代码重构能力的更严格评估，推动更强大的AI编码代理的发展。
排序理由 该集群描述了一个用于评估AI编码代理的新学术基准。
在 Hugging Face Daily Papers 阅读 →
AI 生成摘要 · Google Gemini · 来自 3 个来源。 我们如何撰写摘要 →
arXiv:2608.09802v1 Announce Type: new Abstract: As AI coding agents take on increasingly complex, long-horizon software engineering tasks, existing benchmarks are rapidly saturating and their evaluation quality has come under serious scrutiny: a recent audit found that nearly 60%…
As AI coding agents take on increasingly complex, long-horizon software engineering tasks, existing benchmarks are rapidly saturating and their evaluation quality has come under serious scrutiny: a recent audit found that nearly 60% of unsolved SWE-bench Verified instances contai…
📄 AI paper of the day: SWE-Bench ProMax: Benchmarking Agents on Large-Scale Multilingual Code Refactoring — 75 upvotes on Hugging Face. This benchmark tests agents on real-world refactoring across languages. See the details: https:// huggingface.co/papers/2608.098 02 # AI # Machi…