New framework evaluates jailbreak attacks for LLM safety improvements

Researchers have introduced A-MESS, a new framework for evaluating jailbreak attacks on large language models from a defender's perspective. This approach focuses on how effectively attacks can be used to improve model safety through red-teaming and training, rather than solely on their success rate in breaking the model. A-MESS estimates AttackSHAP scores to attribute marginal utility to individual attacks and select optimal subsets for safety improvements, demonstrating that this defender-centric utility is only weakly aligned with traditional attacker-centric metrics. AI

IMPACT This research reframes the evaluation of jailbreak attacks, potentially leading to more effective safety training data selection for LLMs.

RANK_REASON Academic paper introducing a new evaluation framework for LLM jailbreak attacks. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• arXiv
• AttackSHAP

• paper
• safety

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →