New S1-Evo framework targets and manipulates LLM agent decision models

Researchers have developed S1-Evo, a novel framework designed to attack and manipulate Typed System-1 (S1) decision models, which are increasingly used in LLM agents for tasks like routing and moderation. Unlike attacks targeting autoregressive models, S1-Evo focuses on S1's contract-based option competition and margin feedback. The framework perturbs untrusted fields to create "evidence competition" with original task evidence, aiming to alter the model's execution path. Tested across four benchmarks and three S1 models, S1-Evo achieved a mean attack success rate (ASR) of 71.99% and a high-confidence ASR of 57.47% on the JEV-1.14 benchmark, leading to up to 24.00% end-to-end agent violations. AI

影响 Introduces a new attack vector for System-1 decision models used in LLM agents, potentially impacting their security and reliability.

排序理由 Academic paper detailing a new attack framework on AI models. [lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.AI 阅读 →

• arXiv
• Hugging Face
• LLM agents
• S1-Evo
• System 1

• 论文
• 安全

AI 生成摘要 · Google Gemini · 来自 1 个来源。 我们如何撰写摘要 →