Two new research papers explore vulnerabilities in retrieval-augmented generation (RAG) systems, particularly concerning knowledge poisoning attacks. The first paper, MM-PoisonRAG, introduces a framework to study these attacks in multimodal RAG, detailing localized and globalized poisoning strategies that can significantly manipulate or corrupt model responses, even with limited attacker access. The second paper, Through the Stealth Lens, focuses on developing attention-aware defenses against RAG poisoning, proposing a method using attention weights to detect anomalous passages and improve robustness against such attacks, while also acknowledging the challenge of creating truly stealthy adversarial injections. AI
IMPACT These studies highlight significant security risks in RAG systems, potentially impacting the reliability and safety of AI applications that rely on external knowledge bases.
RANK_REASON The cluster contains two academic papers detailing novel research on AI security vulnerabilities and defenses.
- Attention-Aware Defenses Against Poisoning in RAG
- Attention-Variance Filter
- Normalized Passage Attention Score
- Sarthak Choudhary
- Attention-Variance Filter (AV Filter)
- Globalized Poisoning Attack (GPA)
- Localized Poisoning Attack (LPA)
- MM-PoisonRAG
- MLLM
- multimodal RAG
- Normalized Passage Attention Score (NPAS)
AI-generated summary · Google Gemini · from 2 sources. How we write summaries →
