LLM Refusal Mechanism Shows "Broken Symmetry", Answer Still Recoverable

Researchers have identified a "broken symmetry" in how large language models (LLMs) refuse to answer prompts. They found that even when an LLM generates a refusal, the correct answer is still recoverable from its internal states through localized interventions. However, reimposing a refusal is a more complex process requiring broader interventions across multiple positions. This suggests that LLM refusal is not a simple on/off switch, and probe recoverability might overestimate the ability to control model behavior for safety and auditing purposes. AI

IMPACT Findings suggest current safety probes may overestimate control over LLM refusals, impacting auditing and alignment efforts.

RANK_REASON Academic paper published on arXiv detailing a new finding about LLM behavior. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• arXiv
• Broken Symmetry in LLM Refusal: Answer Release Is More Local Than Refusal Restoration
• Hugging Face

• paper
• safety

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →