A new research paper introduces GUISE, a benchmark designed to evaluate how effectively large language models (LLMs) refuse harmful requests when they are embedded within narrative wrappers. The study found that models like Qwen3-1.7B are highly susceptible to these wrappers, with refusal rates dropping significantly across different languages and registers. To address this vulnerability, the researchers developed AXIS, a defense mechanism that combines preference optimization with objectives to align harmful request representations with refusal directions and ensure complete refusal, achieving high safety and usability scores on Qwen3-1.7B, Qwen3-4B, and GLM-4-9B. AI
IMPACT Highlights a critical safety vulnerability in LLMs and proposes a defense, potentially influencing future model alignment strategies.
RANK_REASON The cluster contains an academic paper detailing a new benchmark and defense mechanism for LLM safety. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
