PulseAugur
中
实时 06:57:16
English(EN) PhysDox: Benchmarking LLMs on Physical Feasibility Auditing of Physiological Sensing Protocols

新基准揭示LLM在医疗协议物理可行性方面存在困难

研究人员开发了PhysDox,一个旨在评估大型语言模型生成的生理传感协议的物理可行性的新基准。该基准包含一个精心策划的Gold集合(包含683个样本)和一个Silver集合(包含5000个样本),涵盖六个生物医学领域。对六个LLM的初步评估显示,最高阶段1宏F1分数仅为53.0,表明其审计协议有效性方面存在显著局限性。分析显示,模型经常将程序完整性与物理可行性混淆,隐式约束被遗漏的频率是显式硬件违规的两倍。 AI

影响 凸显了LLM在生物医学协议设计等复杂现实世界应用中推理能力的局限性。

排序理由 该集群描述了一篇介绍LLM评估基准的新学术论文。[lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.AI 阅读 →

AI 生成摘要 · Google Gemini · 来自 1 个来源。 我们如何撰写摘要 →

新基准揭示LLM在医疗协议物理可行性方面存在困难

本文如何被排名

Signal score
37 / 100
Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket
Tool
该集群描述了一篇介绍LLM评估基准的新学术论文。[lever_c_demoted from research: ic=1 ai=1.0]
Source corroboration
Single-source cluster
Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics
paper, other
Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance
High
Clearly on-topic for AI-industry coverage.
Story freshness
Breaking (< 6h)
Fresh story with cross-source coverage still developing. Ranking may shift as more sources report.

完整方法见我们的编辑标准。

报道来源 [1]

  1. arXiv cs.AI TIER_1 English(EN) · He Liu, Boyuan Gu, Shuaiqi Cheng, Haiyang Sun, Siyu You, Xuming Hu ·

    PhysDox:在生理传感协议的物理可行性审计方面对大型语言模型进行基准测试

    arXiv:2606.05003v2 Announce Type: cross Abstract: Large language models (LLMs) increasingly assist in experimental design, yet fluent protocols often remain physically infeasible. We introduce PhysDox, a physical feasibility auditing benchmark for biomedical protocols comprising …