一项新研究表明，旨在解释大型语言模型（LLM）思考过程的自然语言自编码器（NLA）在初始化误差方面出奇地鲁棒。研究人员发现，即使使用完全不合理的陈述进行初始化，NLA 也能达到很高的重建准确率，尽管它们的解释在很大程度上仍然是无意义的。这表明 NLA 在理解 LLM 推理方面的效用可能有限，因为它们的输出与准确的内部状态没有可靠的联系。 AI
影响 对目前解释 LLM 内部状态的可靠性方法提出了质疑。
排序理由 研究论文，详细介绍了关于自然语言自编码器鲁棒性的发现。
AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →
<p>Natural language autoencoders are meant to take in an LLM's activation vector and describe in plain text what the model is thinking. However, its training data collection involves asking Claude to guess what a model might be thinking. How robust are NLAs to these guesses? We c…
<p>Natural language autoencoders are meant to take in an LLM's activation vector and describe in plain text what the model is thinking. However, its training data collection involves asking Claude to guess what a model might be thinking. How robust are NLAs to these guesses? We c…