English(EN) PAYWORTHY: Can LLMs Predict Which Freelance Jobs an AI Agent Will Complete?

大型语言模型可以预测AI代理工作的成功率，但存在特定限制

作者 PulseAugur 编辑部 · [1 个来源] · 2026-11-03 23:59

研究人员开发了一个名为PAYWORTHY的新基准，用于评估大型语言模型（LLMs）能否预测AI代理在自由职业工作中的成功率。该基准包含1,494个真实的自由职业职位发布，LLMs的任务是预测代理成功的可能性。当提供代理的具体限制时，大多数模型表现出强大的预测能力，AUC得分在0.66到0.86之间。然而，当被描述为拥有标准工具的通用代理时，这些模型表现与随机猜测无异。 AI

影响这项研究强调了特定代理能力对于成功完成任务的重要性，并表明大型语言模型可以被训练来评估这些能力。

排序理由该集群描述了一篇介绍LLM评估基准的新学术论文。[lever_c_demoted from research: ic=1 ai=1.0]

在 Hugging Face Daily Papers 阅读 →

AI 生成摘要 · Google Gemini · 来自 1 个来源。我们如何撰写摘要 →

本文如何被排名

Signal score: 0 / 100; Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket: Tool; 该集群描述了一篇介绍LLM评估基准的新学术论文。[lever_c_demoted from research: ic=1 ai=1.0]
Source corroboration: Single-source cluster; Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics: paper, model release; Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance: High; Clearly on-topic for AI-industry coverage.
Story freshness: 6 days old; Aged out of breaking-news scoring windows; ranking reflects the durable signal from the full source set.

完整方法见我们的编辑标准。

报道来源 [1]

Hugging Face Daily Papers TIER_1 English(EN) · 2026-11-03 23:59

PAYWORTHY：大型语言模型能否预测AI代理将完成哪些自由职业工作？

Before an AI agent takes on a paid job, someone has to judge whether it is likely to succeed. We test whether frontier language models can make this call before any attempt. We propose PAYWORTHY which contains 1{,}494 real freelance job postings, sampled from 18.8M postings made …

本文如何被排名

报道来源 [1]

PAYWORTHY：大型语言模型能否预测AI代理将完成哪些自由职业工作？

相关实体

相关话题