Steering vectors in LLMs found to be an attack surface

Researchers have identified a new vulnerability in activation steering techniques used to control Large Language Models. By subtly poisoning steering datasets with a small percentage of malicious tokens, an attacker can create vectors that jailbreak models while preserving their intended function. This stealth attack can achieve a significant success rate in bypassing safety mechanisms, though a proposed orthogonalization defense shows promise in mitigating the threat. AI

IMPACT Highlights a novel attack vector against LLM safety mechanisms, potentially impacting the deployment of steerable models.

RANK_REASON Academic paper detailing a new security vulnerability in LLM control techniques. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• activation steering
• Donato Crisostomi
• Large Language Model

• paper
• safety

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →