New defense filters RAG poisoning using LLM attention weights

Researchers have developed a new defense mechanism called the Attention-Variance Filter (AV Filter) to protect Retrieval-Augmented Generation (RAG) systems from poisoning attacks. These attacks inject malicious passages into the RAG system's context, even at low corruption rates, to manipulate responses. The AV Filter utilizes attention weights from large language models to identify anomalous passages, improving accuracy by up to 20% over existing defenses. While adaptive attacks can achieve a 35% success rate in concealing these anomalies, the research highlights the ongoing challenges in achieving true stealth for RAG poisoning. AI

Summary written by gemini-2.5-flash-lite from 1 sources. How we write summaries →

IMPACT Enhances RAG system security by introducing a novel defense against data poisoning attacks.

RANK_REASON The cluster contains an academic paper detailing a new method for improving AI system security. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• Attention-Aware Defenses Against Poisoning in RAG
• Attention-Variance Filter
• Normalized Passage Attention Score
• Retrieval-Augmented Generation
• Sarthak Choudhary

• paper
• safety