Classification suffixes boost LLM agent detection of malicious inputs

Researchers have investigated the effectiveness of using classification suffixes to improve the detection of malicious inputs by LLM agents. Their study, which tested various suffixes across 13 safety benchmarks and three open-weight model families, found that appending a classification instruction consistently enhances out-of-distribution detection compared to no suffix. The wording of the suffix matters, with classification prompts proving more effective than off-topic or merely attentive ones. This benefit stems from the classification format itself, with the specific criterion adding precision only at stricter thresholds. The findings suggest that these classification suffixes, served via KV-cache forks, can be a cost-effective addition to activation-probe monitors, though their efficacy depends on the specific model and readout method. AI

IMPACT Enhances LLM safety monitoring by improving the detection of malicious inputs with minimal cost.

RANK_REASON Academic paper detailing a novel method for improving LLM safety probes. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• arXiv
• Gemma 4-12B
• KV cache
• Llama-3.1:8b
• multilayer perceptron
• Qwen3.5:9b

• paper
• safety

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →