New method detects adversarial LLM prompts using sequential entropy changes

Researchers have developed a new method called CPD Online to detect adversarial prompts that attempt to jailbreak large language models. This technique treats prompt detection as an online change-point detection problem, analyzing sequential entropy changes in the model's token predictions. CPD Online is model-agnostic, requires no training, and can pinpoint the onset of malicious prompts, outperforming existing perplexity-based detectors on various open-weight models. AI

影响 This new detection method could enhance the safety of LLMs by identifying and mitigating malicious prompts, potentially reducing the need for extensive guardrail interventions.

排序理由 The cluster contains a new academic paper detailing a novel method for detecting adversarial prompts in LLMs. [lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.AI 阅读 →

• CPD Online
• LLaMA-2-7B
• LLaMA Guard
• LLMs
• Mohammed Alshaalan

• 论文
• 安全

AI 生成摘要 · Google Gemini · 来自 1 个来源。 我们如何撰写摘要 →