English(EN) SWE-Journey: Towards More Realistic Evaluation of Coding Assistants through Long-Horizon, Multi-Turn Interaction

新的SWE-Journey基准测试评估编码助手在真实任务上的表现

作者 PulseAugur 编辑部 · [1 个来源] · 2026-10-08 09:21

研究人员推出SWE-Journey，这是一个旨在更真实地评估Claude Code和Codex等编码助手的新基准测试。该基准测试通过关注长时程任务和多轮交互来解决现有评估的局限性，这对于实际软件开发至关重要。该系统使用弱到强合成流水线来创建扩展编码任务，并基于四种不同的用户画像模拟真实的交互。初步结果表明，虽然助手在软件架构师场景下表现良好，但在与非编码人员交互时却面临显著困难，这凸显了它们在支持技术专业知识较少的用户方面的能力差距。 AI

影响该基准测试通过突出当前在处理复杂、交互式任务方面的局限性，有望促使开发出更强大、更用户友好的编码助手。

排序理由该项目是一篇研究论文，介绍了一个用于评估AI编码助手的新基准测试。[lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.MA (Multiagent) 阅读 →

AI 生成摘要 · Google Gemini · 来自 1 个来源。我们如何撰写摘要 →

本文如何被排名

Signal score: 3 / 100; Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket: Tool; 该项目是一篇研究论文，介绍了一个用于评估AI编码助手的新基准测试。[lever_c_demoted from research: ic=1 ai=1.0]
Source corroboration: Single-source cluster; Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics: paper, product; Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance: High; Clearly on-topic for AI-industry coverage.
Story freshness: Same-day; Cluster formed today. Ranking reflects the current source set at time of score.

完整方法见我们的编辑标准。

报道来源 [1]

arXiv cs.MA (Multiagent) TIER_1 English(EN) · Xuebo Liu · 2026-10-08 09:21

SWE-Journey：通过长时程、多轮交互实现更真实的编码助手评估

Coding assistants such as Claude Code and Codex have become a major application of LLM agents, yet existing benchmarks remain far from real-world use, particularly in task horizon and interaction length. Code assistants require completing long chains of development work in contin…

本文如何被排名

报道来源 [1]

SWE-Journey：通过长时程、多轮交互实现更真实的编码助手评估

相关实体

相关话题