New DenialRAG attack poisons RAG systems by embedding false answers

Researchers have developed a new method called DenialRAG to poison retrieval-augmented generation (RAG) systems by embedding a denial of the correct answer within a crafted document. This attack explicitly names the correct answer, refutes it, and provides a fabricated explanation for an incorrect one, directly conflicting with the true information presented to the LLM. Evaluations across multiple datasets, LLMs including Mistral-7B, and various defenses demonstrated that DenialRAG is highly effective on certain models, particularly Mistral-7B, and that RAG poisoning risk is complex and model-dependent. AI

IMPACT Highlights a new vulnerability in RAG systems, potentially impacting the reliability of AI applications that rely on them.

RANK_REASON Academic paper detailing a new attack method on RAG systems. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• DenialRAG
• large language model
• mistral:7b
• retrieval-augmented generation

• paper
• safety

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →