Researchers have introduced DSPrompt, a novel defense framework designed to protect Multimodal Retrieval Augmented Generation (M-RAG) systems from adversarial attacks. Unlike existing methods that add inference overhead, DSPrompt dynamically reshapes the retriever's embedding semantics by inserting learnable soft prompts into the visual and textual encoders. This approach trains the system using a min-max scheme where an attacker crafts adversarial documents, and the defender learns to push them out of the top-k rankings while preserving benign data. Experiments show DSPrompt significantly reduces attack success rates with minimal computational cost and parameter addition, outperforming current defense baselines. AI
IMPACT Introduces a novel defense mechanism for M-RAG systems, potentially improving their robustness against adversarial attacks without significant computational overhead.
RANK_REASON The item describes a new research paper detailing a novel defense framework for M-RAG systems. [lever_c_demoted from research: ic=1 ai=1.0]
Read on Hugging Face Daily Papers →
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
