LLM harmfulness representations evolve across multi-turn attacks

Researchers have investigated the geometric and temporal dynamics of harmfulness and refusal representations in large language models during multi-turn attacks. Analyzing models like Llama 3.1 8B-Instruct, Qwen2.5-7B-Instruct, and Gemma-2-9B-it under various attack frameworks, they found that harmfulness representations become increasingly separable across conversation turns, particularly at the end-of-turn token. These harmfulness representations showed only weak alignment with refusal-related representations, suggesting that multi-turn attacks succeed not by suppressing harmfulness internally, but by exploiting its evolving separability over time. The findings imply that future safety defenses should account for these temporal dynamics rather than relying solely on single-turn probes. AI

IMPACT Suggests new directions for LLM safety research by highlighting the temporal dynamics of harmfulness representations in multi-turn attacks.

RANK_REASON Academic paper detailing novel research findings on LLM safety. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• ActorAttack
• arXiv
• crescendo
• Gemma-2-9B-it
• Hugging Face
• Llama 3.1 8B-Instruct
• Qwen2.5-7B-Instruct
• X-Teaming

• paper
• safety

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →