New method uses model's own outputs for safety fine-tuning

Researchers have developed a novel method for safety fine-tuning language models by identifying and utilizing the most challenging prompts. This technique involves scoring prompts based on the frequency of harmful model responses and then training on these difficult prompts using the model's own non-jailbroken outputs. Initial tests on Llama-3 models showed a significant reduction in attack success rates, though it also increased the model's tendency to refuse benign prompts. Further adjustments, including interleaving with adversarially-framed benign prompts and focusing on the hardest eligible prompts, helped mitigate this refusal issue while maintaining strong safety performance. AI

IMPACT Introduces a new technique for improving LLM safety that could reduce the effectiveness of jailbreaking attacks.

RANK_REASON Academic paper detailing a new method for safety fine-tuning language models. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• arXiv
• Llama-3.2-3B-Instruct
• Llama-3-8B-Instruct

• paper
• safety

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →