New GOAL framework tackles over-refusal in safety-aligned LLMs

A new research paper introduces GOAL (Gradient Orthogonalization with contrAstive Learning), a framework designed to address the issue of over-refusal in safety-aligned large language models (LLMs). Over-refusal occurs when LLMs mistakenly reject harmless queries due to their safety alignment. The GOAL framework analyzes this problem from representation and optimization perspectives, noting that harmful and over-refusal sample representations become indistinguishable in deeper layers and that harmful gradients align more strongly with refusal targets. GOAL aims to mitigate over-refusal by orthogonalizing over-refusal gradients against harmful gradients and using contrastive learning to enhance representation separation, demonstrating effectiveness across multiple models and benchmarks. AI

IMPACT This research could lead to LLMs that are better at distinguishing between harmful and harmless queries, improving user experience and utility.

RANK_REASON The cluster contains an academic paper detailing a new method for improving LLM safety alignment. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• arXiv
• GOAL
• Gradient Orthogonalization with contrAstive Learning
• Hugging Face
• large-language models

• paper
• safety

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →