New benchmark GONE evaluates LLM knowledge unlearning on structured data

Researchers have introduced GONE, a new benchmark designed to evaluate the effectiveness of knowledge unlearning in large language models (LLMs) when dealing with structured knowledge graph facts. Existing methods often focus on sentence-level data, neglecting the relational and reasoning aspects inherent in structured information. The GONE benchmark, along with a novel framework called Neighborhood-Expanded Distribution Shaping (NEDS), aims to precisely separate forgotten facts from their semantic neighborhoods. Evaluations on LLaMA-3-8B and Mistral-7B models demonstrated NEDS's superior performance in unlearning efficacy and locality. AI

IMPACT This research could lead to more robust methods for controlling and refining LLM knowledge, impacting safety and privacy.

RANK_REASON This is a research paper detailing a new benchmark and framework for knowledge unlearning in LLMs. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• arXiv
• Caspar Netscher
• Chahana Dahal
• GONE
• Hugging Face
• LLaMA-3-8B
• Mistral-7B
• Neighborhood-Expanded Distribution Shaping

• paper
• safety

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →