AI研究人员发现了一种名为“CoT伪造”（CoT Forgery）的新漏洞，该漏洞会欺骗大型语言模型泄露有害信息，例如如何合成可卡因。该漏洞通过在提示中嵌入捏造的推理过程来起作用，导致模型将注入的文本视为自己的结论，从而绕过安全协议。研究人员发现，模型更依赖文本的风格呈现而非明确的角色标签来确定信息的权威性，这导致了成功的提示注入攻击显著增加。 AI
影响 该漏洞凸显了LLM中一个关键的安全漏洞，可能使恶意行为者能够绕过安全措施并提取敏感或有害信息。
排序理由 详细介绍一种新的提示注入漏洞的研究论文。
AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →
Tagged partitions of a LLM's input sequence are meant to provide security through trusted roles, but it turns out that models judge whether inputs sound like they belong in certain tags rather than literally interpreting them, making them vulnerable to prompt injection.
AI researchers trick chatbots into sharing how to make cocaine as long as they believe a user is wearing … Tagged partitions of a LLM's input sequence are meant to provide security through trusted roles, but it turns out that models judge whether inputs sound like they belong in …