New framework tests LLM disinformation detectors against adversarial attacks

Researchers have developed a new framework called Build it, Break it, Repeat (BiBiR) to test the robustness of disinformation detection models against LLM-manipulated content. This iterative approach simulates adversarial conditions where disinformation posts are systematically altered to evade classification. In experiments, the best adversarial transformations involved back-translation and LLM persona-based rewriting, achieving a 95% label flip rate while preserving the original meaning. The top-performing detection model, a triplet contrastive architecture with dynamic anchor switching (DASS), achieved 72.68% accuracy against these sophisticated attacks, significantly outperforming a fine-tuned e5-small-LoRA baseline. AI

IMPACT This research highlights the need for more robust evaluation methods for AI-driven disinformation detection, crucial for maintaining platform integrity.

RANK_REASON Academic paper detailing a new methodology for evaluating AI model robustness. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• Build it, Break it, Fix it
• Build it, Break it, Repeat
• Carolina Scarton
• DASS
• e5-small-LoRA

• paper
• safety

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →