A Black‑Box Assessment of LlamaGuard’s Robustness to RAG Injection Attacks
A security researcher found that LlamaGuard-3-1B, a model designed to protect against harmful content, completely failed to detect 10 different RAG injection attacks. These attacks, which have previously succeeded against other LLMs, were all classified as safe by LlamaGuard. In contrast, a smaller model called PromptGuard-86M successfully identified all the injection attempts, highlighting a critical difference in how these models are trained and their effectiveness against instruction integrity issues rather than just content safety. AI
IMPACT Highlights critical vulnerabilities in current AI safety models, suggesting a need for specialized defenses against instruction integrity attacks.