Through the Stealth Lens: Attention-Aware Defenses Against Poisoning in RAG
Researchers have developed a new defense mechanism called the Attention-Variance Filter (AV Filter) to protect Retrieval-Augmented Generation (RAG) systems from poisoning attacks. These attacks inject malicious passages into the RAG system's context, even at low corruption rates, to manipulate responses. The AV Filter utilizes attention weights from large language models to identify anomalous passages, improving accuracy by up to 20% over existing defenses. While adaptive attacks can achieve a 35% success rate in concealing these anomalies, the research highlights the ongoing challenges in achieving true stealth for RAG poisoning. AI
IMPACT Enhances RAG system security by introducing a novel defense against data poisoning attacks.