English(EN)Can CaMeLs Talk? Securing Multi-Agent Systems Against Indirect Prompt Injection Attacks
新的 AI 护栏应对不断演变的提示注入威胁
作者PulseAugur 编辑部·[9 个来源]·
研究人员正在开发新的方法来保护 AI 智能体免受复杂的提示注入攻击,这些攻击将恶意指令嵌入到智能体处理的内容中。一种名为 VEPA 的方法使用一个随着部署反馈而演变的正式策略框架,以确保确定性和可验证的护栏决策。另一种技术 BRANCH 采用分支树搜索,以更少的查询绕过多扫描仪护栏系统。此外,AdaGuard 利用 LLM-as-a-Judge 框架进行自适应推理,以增强安全性和策略合规性,而 multi-CaMeL 通过在智能体边界之间保留数据来源来保护多智能体系统。这些进展旨在防止 AI 智能体执行意外或有害的操作,即使面对间接注入方法。
AI
影响
通过开发针对复杂提示注入和间接攻击的强大防御措施,增强了 AI 智能体的安全性,这对于安全部署至关重要。
arXiv:2610.18304v1 Announce Type: new Abstract: Guardrail approaches that rely on reasoning-based large language models (LLM) improve safety decisions, but come with substantial inference overhead due to the explicit reasoning. We introduce LR-Guard (Latent Reasoning Guardrail), …
arXiv cs.AI
TIER_1English(EN)·Duy Nguyen, Lesly Miculicich, Bhavana Dalvi Mishra, Shuo Li, Krishnamurthy Dj Dvijotham, Mohit Bansal, Tomas Pfister, Long T. Le·
arXiv:2610.12813v1 Announce Type: new Abstract: LLM agents increasingly operate over sensitive resources, such as private documents and mutable system states, which can create serious safety and security risks. Runtime guardrails should therefore make inspectable, reproducible, a…
arXiv cs.AI
TIER_1English(EN)·William Hackett, Peter Garraghan·
arXiv:2610.10742v1 Announce Type: cross Abstract: AI systems increasingly rely on Large Language Models (LLMs) as core reasoning engines, making them targets for prompt injection and jailbreaks. Guardrails monitor and validate model inputs and outputs, yet their isolated, task-fo…
Enterprise generative AI applications require robust safety mechanisms that can accommodate diverse risk postures, evolving policies, and varying latency constraints. Current guardrail solutions often suffer from rigidity, relying on fixed policy sets and offering limited transpa…
arXiv cs.MA (Multiagent)
TIER_1English(EN)·Christian Schroeder de Witt·
Indirect prompt injection attacks - malicious instructions embedded in content processed by large language models - remain a major obstacle to safely deploying tool-using agents. CaMeL [Debenedetti et al., 2025] mitigates this threat for an individual agent by separating trusted …
<p>In February 2024, a Canadian tribunal ordered Air Canada to honor a bereavement refund policy that its support chatbot had invented. The airline argued the bot was responsible for its own words. The tribunal disagreed. If your agent can say it, or do it, you own it.</p> <p>The…
<p>In Part 1, we looked at the attack from the attacker's perspective.</p> <p>The attacker does not necessarily need to send a malicious prompt to the AI. They may only need to place malicious instructions inside something the AI already reads: an email, document, web page, ticke…
<p>description: "How attackers can compromise AI agents without ever touching the AI interface—by hiding instructions inside documents, emails, web pages, RAG content, and tool responses."</p> <h2> The Trust Propagation Layer </h2> <p>The critical failure in a 0-click attack is n…
AI coding agents introduce a new security challenge: prompt injection that can hide from traditional guardrails. Researchers at Adversa AI demonstrated a technique called Cryptographic Context Injection (CCI) that can potentially trick GitHub Copilot CLI into executing malicious …