Steering Vectors are an Adversarial Attack Surface
Researchers have identified a new vulnerability in activation steering techniques used to control Large Language Models. By subtly poisoning steering datasets with a small percentage of malicious tokens, an attacker can create vectors that jailbreak models while preserving their intended function. This stealth attack can achieve a significant success rate in bypassing safety mechanisms, though a proposed orthogonalization defense shows promise in mitigating the threat. AI
IMPACT Highlights a novel attack vector against LLM safety mechanisms, potentially impacting the deployment of steerable models.