Researchers have introduced SWARM, a new multilingual dataset designed to detect Russian propaganda within search engine results. The dataset comprises 2,183 search engine results across nine languages, with each entry annotated by trained coders to identify support for recurring Russian propaganda narratives. Benchmarking efforts using a source-based blocklist, supervised classifiers, and large language models (LLMs) revealed that blocklists are insufficient as propaganda appears on mainstream sites, not just flagged outlets. While content-level analysis is more effective, LLMs achieved a higher F1 score of 0.73 compared to supervised classifiers at approximately 0.5, with smaller LLMs sometimes mistaking topical relevance for endorsement. AI
IMPACT This dataset could improve the ability of AI models to identify and mitigate the spread of state-sponsored disinformation across different languages and online platforms.
RANK_REASON The cluster contains a research paper introducing a new dataset and evaluation methodology. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
