Researchers have developed a new evaluation protocol called LENS (Level-based Evaluation of Narrative Suppression) to assess the effectiveness of machine unlearning in preventing large language models from reproducing disinformation-aligned narrative frames. The study tested four instruction-tuned models—Lapa LLM, Gemma-12B, Qwen-14B, and TAIDE-Gemma—using two specific narratives related to the Russia-Ukraine conflict and the US-Taiwan relationship. Results indicated that while unlearning could reduce narrative reproduction, it also led to unintended side effects like entity recovery, demonstrating the complexity of narrative unlearning. AI
IMPACT Introduces a new method for evaluating and potentially improving the safety of LLMs against disinformation.
RANK_REASON The cluster contains a research paper detailing a new evaluation protocol for LLM safety. [lever_c_demoted from research: ic=1 ai=1.0]
- Gemma-12B
- Lapa LLM
- Large language models
- LENS
- Qwen-14B
- Russia
- TAIDE-Gemma
- Taiwan
- Ukraine
- United States
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
