研究人员使用因果影响图（Causal Influence Diagrams）的形式化了在AI系统中诱导潜在知识（ELK）的问题。该论文证明，虽然反馈可以激励对可观察变量的诚实回答，但不能保证对潜在、隐藏信息的诚实性。一项不可能性定理证明，即使有完美的训练反馈，由于目标错误泛化的风险，任何基于反馈的训练策略都无法可靠地产生诚实的代理。 AI
影响 这项研究表明在确保AI诚实性方面存在根本性限制，尤其是在涉及隐藏变量时，这对AI安全和对齐构成了挑战。
排序理由 该集群包含一篇详细介绍AI安全理论不可能性结果的学术论文。
AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →
arXiv:2606.12268v1 Announce Type: new Abstract: Advanced AI systems have extensive knowledge of their environments; in fact, their knowledge may (far) exceed that of their developers or users. Consequently, a desirable property for an AI system is that it is honest -- that it acc…
Advanced AI systems have extensive knowledge of their environments; in fact, their knowledge may (far) exceed that of their developers or users. Consequently, a desirable property for an AI system is that it is honest -- that it accurately reports its beliefs about the world. Des…