Uncensored LLMs slash safety refusals but lose capabilities

A recent analysis compared three pairs of language models, evaluating their safety refusals and helpfulness. The study found that "uncensored" versions of models like Llama 3.1 8B, Qwen2.5 7B, and Mistral 7B Instruct significantly reduced harmful refusals, dropping from 80-100% to 21-34%. However, this reduction in safety did not lead to improved helpfulness on benign prompts, with uncensored models scoring lower in some cases. Furthermore, the retention of core capabilities varied greatly, with one pair showing a slight gain while others experienced substantial losses, indicating that uncensoring is not a simple upgrade but rather a modification that alters a model's behavior and potential risks. AI

IMPACT Uncensoring LLMs significantly reduces safety refusals but may compromise helpfulness and capability, suggesting a trade-off for developers.

RANK_REASON Analysis of model behavior and capabilities based on a benchmark. [lever_c_demoted from research: ic=1 ai=1.0]

Read on dev.to — LLM tag →

• DeepSeek V4-Pro
• Dolphin Mistral 7B
• huihui_ai Qwen2.5 7B Abliterate
• lforla
• Llama 3.1 8B Instruct
• Mannix Llama 3.1 8B Abliterated
• Mistral 7B Instruct v0.3
• Nvidia T4
• Ollama

• model release
• safety

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →