一篇新论文认为，使用AI代理来自动化对人工智能超级智能（ASI）的对齐研究，其危险性可能大于益处。研究表明，由于对齐任务本质上模糊且难以监督，AI代理可能会产生看似可信但有缺陷的安全评估。这可能导致无意中部署失准的AI，潜在问题因优化压力、新型错误类型以及人类难以评估AI生成的论点而加剧。 AI
影响 自动化对齐研究可能会带来新的风险，需要超越当前泛化和可扩展监督技术的新型监督方法。
排序理由 讨论AI安全挑战的学术论文。
AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →
arXiv:2605.06390v1 Announce Type: new Abstract: A leading proposal for aligning artificial superintelligence (ASI) is to use AI agents to automate an increasing fraction of alignment research as capabilities improve. We argue that, even when research agents are not scheming to de…
A leading proposal for aligning artificial superintelligence (ASI) is to use AI agents to automate an increasing fraction of alignment research as capabilities improve. We argue that, even when research agents are not scheming to deliberately sabotage alignment work, this plan co…