English(EN)New framework for auditing machine unlearning
新研究应对大型语言模型中的机器遗忘挑战
作者PulseAugur 编辑部·[14 个来源]·
研究人员正在开发用于大型语言模型（LLM）的机器遗忘新方法，这一过程对于隐私和知识管理至关重要。多篇论文探讨了在不完全重新训练的情况下从已训练模型中移除特定数据的技术。这些技术包括用于混合专家（Mixture-of-Experts）模型的TRACE、用于平滑预测概率的LoTUS以及用于学习token级别重要性的ATWU。其他研究则调查了遗忘的最佳实践，例如使用多样化的邻居集和模块化采样，并强调了多个训练种子对于可靠评估的重要性。新发现的一个挑战是遗忘痕迹的可检测性，这些痕迹可能残留在模型的输出和内部表示中。
AI
arXiv:2606.12809v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) are trained on massive multimodal data, making data unlearning increasingly important as data owners may request the removal of specific content. In practice, these requests often arrive sequ…
arXiv:2606.10338v1 Announce Type: cross Abstract: Machine unlearning is increasingly important for large language models, yet unlearning in Mixture-of-Experts (MoE) architectures remains underexplored. Unlike dense models, MoE architectures employ a router at each layer to assign…
arXiv:2503.18314v5 Announce Type: replace-cross Abstract: We present LoTUS, a novel Machine Unlearning (MU) method that eliminates the influence of training samples from pre-trained models, avoiding retraining from scratch. LoTUS smooths the prediction probabilities of the model …
Machine unlearning is increasingly important for large language models, yet unlearning in Mixture-of-Experts (MoE) architectures remains underexplored. Unlike dense models, MoE architectures employ a router at each layer to assign each token to a sparse subset of experts. In this…
arXiv cs.AI
TIER_1English(EN)·Jamie Lanyon, Axel Finke, Petros Andreou, Georgina Cosma·
arXiv:2510.26714v5 Announce Type: replace-cross Abstract: Machine unlearning aims to remove the influence of certain data points from a trained model without costly retraining. Most practical unlearning algorithms are only approximate and their performance can only be assessed em…
arXiv:2509.05316v2 Announce Type: replace-cross Abstract: A conventional LLM Unlearning setting consists of two subsets -"forget" and "retain", with the objectives of removing the undesired knowledge from the forget set while preserving the remaining knowledge from the retain. In…
arXiv:2412.09119v3 Announce Type: replace Abstract: Machine unlearning, the process of selectively removing data from trained models, is increasingly crucial for addressing privacy concerns and knowledge gaps post-deployment. Despite this importance, existing approaches are often…
arXiv cs.CL
TIER_1English(EN)·Gizem Y\"uce, Giorgos Nikolaou, Nicolas Flammarion·
arXiv:2606.06320v1 Announce Type: cross Abstract: Machine unlearning aims to remove targeted knowledge from a trained model while preserving its general capabilities. For autoregressive language models, not all tokens in a forget sample are equally relevant to forgetting. Existin…
Machine unlearning aims to remove targeted knowledge from a trained model while preserving its general capabilities. For autoregressive language models, not all tokens in a forget sample are equally relevant to forgetting. Existing approaches either ignore this heterogeneity or r…
arXiv cs.LG
TIER_1English(EN)·Ahmed Mehdi Inane, Vincent Quirion, Gintare Karolina Dziugaite, Ioannis Mitliagkas·
arXiv:2605.11170v2 Announce Type: replace Abstract: Noise-based certified machine unlearning currently faces a hard ceiling: the noise magnitude required to certify unlearning typically destroys model utility, particularly for large-scale deletion requests. While leveraging publi…
arXiv cs.CL
TIER_1English(EN)·Anna Borisiuk, Andrey Savchenko, Alexander Panchenko, Elena Tutubalina·
arXiv:2602.19612v5 Announce Type: replace Abstract: Machine Unlearning (MU) enables Large Language Models (LLMs) to remove unsafe or outdated information. However, existing work assumes that all facts are equally forgettable and largely ignores whether the forgotten knowledge ori…
arXiv:2506.14003v5 Announce Type: replace Abstract: Machine unlearning (MU) for large language models (LLMs), commonly referred to as LLM unlearning, seeks to remove specific undesirable data or knowledge from a trained model, while maintaining its performance on standard tasks. …
arXiv cs.AI
TIER_1English(EN)·SeungBum Ha, Saerom Park, Sung Whan Yoon·
arXiv:2506.01318v4 Announce Type: replace-cross Abstract: Machine unlearning (MU) aims to expunge a designated forget set from a trained model without costly retraining, yet the existing techniques overlook two critical blind spots: "over-unlearning" that deteriorates retained da…
