English(EN) SWE-IF: Aligning Code Evaluation with Human Preference

新的SWE-IF框架评估LLM在代码指令遵循方面的能力

作者 PulseAugur 编辑部 · [1 个来源] · 2026-06-08 04:00

研究人员推出SWE-IF，这是一个新的评估框架，旨在超越功能正确性来评估大型语言模型（LLM）遵循代码指令的能力。该框架包含一个包含30个可验证代码指令的分类法和确定性验证器，旨在捕捉反映人类对干净、保留意图和正确代码偏好的“氛围检查”。对31个LLM的评估显示，指令遵循是一个关键的区分因素，功能正确性和指令遵循的综合得分与人类偏好最相关。 AI

影响这个新的评估框架可能促使LLM生成更符合人类偏好且更易于维护的代码，从而提高开发者的生产力。

排序理由该集群包含一篇介绍LLM新评估框架的学术论文。[lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.AI 阅读 →

AI 生成摘要 · Google Gemini · 来自 1 个来源。我们如何撰写摘要 →

本文如何被排名

Signal score: 0 / 100; Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket: Tool; 该集群包含一篇介绍LLM新评估框架的学术论文。[lever_c_demoted from research: ic=1 ai=1.0]
Source corroboration: Single-source cluster; Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics: paper, model release; Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance: High; Clearly on-topic for AI-industry coverage.
Story freshness: 114 days old; Aged out of breaking-news scoring windows; ranking reflects the durable signal from the full source set.

完整方法见我们的编辑标准。

报道来源 [1]

arXiv cs.AI TIER_1 English(EN) · Ming Zhong, Xiang Zhou, Ting-Yun Chang, Qingze Wang, Nan Xu, Xiance Si, Dan Garrette, Shyam Upadhyay, Jeremiah Liu, Jiawei Han, Benoit Schillings, Jiao Sun · 2026-06-08 04:00

SWE-IF：代码评估与人类偏好对齐

arXiv:2510.07315v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) have catalyzed vibe coding, where users leverage LLMs to generate and iteratively refine code through natural language interactions until it passes their vibe check. Vibe check reflects human p…

本文如何被排名

报道来源 [1]

SWE-IF：代码评估与人类偏好对齐

相关实体

相关话题