New DisarmRAG attack disables LLM self-correction in RAG systems

Researchers have developed a new method called DisarmRAG to exploit vulnerabilities in retrieval-augmented generation (RAG) systems. Unlike previous attacks that targeted the knowledge base, DisarmRAG compromises the retriever component to inject instructions that disable the large language model's self-correction ability. This novel approach utilizes an iterative co-optimization process and a stealthy model editing technique to ensure effectiveness, achieving over 90% success rates across multiple LLMs and benchmarks, even against detection defenses. AI

IMPACT Highlights a new vulnerability in RAG systems, potentially impacting the reliability and security of LLM deployments.

RANK_REASON Academic paper detailing a novel attack method on LLM systems. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• DisarmRAG
• large language models
• retrieval-augmented generation
• Yanbo Dai

• paper
• safety

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →