PulseAugur
实时 17:48:01

新研究应对 LLM 中的机器遗忘挑战

研究人员正在开发大型语言模型(LLM)中机器遗忘的新方法,这一过程对于隐私和知识管理至关重要。几篇论文探讨了在不完全重新训练的情况下从训练模型中删除特定数据的技术。这些技术包括用于专家混合(Mixture-of-Experts)模型的 TRACE、用于平滑预测概率的 LoTUS 以及用于学习 token 级别重要性的 ATWU。其他研究则关注遗忘的最佳实践,例如使用多样化的邻居集和模块化采样,并强调了多个训练种子对于可靠评估的重要性。新发现的一个挑战是遗忘痕迹的可检测性,这些痕迹可能在模型输出和内部表示中持续存在。 AI

影响 机器遗忘技术的进步对于增强 LLM 的隐私、安全性和适应性至关重要,从而能够实现更负责任的部署。

排序理由 多篇学术论文在 arXiv 上发表,详细介绍了 LLM 机器遗忘的新方法和分析。

在 Google AI / Research 阅读 →

AI 生成摘要 · Google Gemini · 来自 14 个来源。 我们如何撰写摘要 →

新研究应对 LLM 中的机器遗忘挑战

报道来源 [14]

  1. Google AI / Research TIER_1 English(EN) ·

    用于审计机器遗忘的新框架

    Algorithms & Theory

  2. arXiv cs.AI TIER_1 English(EN) · He Li, Haoang Chi, Qizhou Wang, Yunxin Mao, Zhiheng Zhang, Jie Tan, Tongliang Liu, Wenjing Yang, Bo Han ·

    MLUBench:MLLMs 终身遗忘评估基准

    arXiv:2606.12809v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) are trained on massive multimodal data, making data unlearning increasingly important as data owners may request the removal of specific content. In practice, these requests often arrive sequ…

  3. arXiv cs.AI TIER_1 English(EN) · Jingyi Xie, Yijun Lin, Yinjiang Xiong, Zhikun Zhang, Sai Li ·

    面向混合专家语言模型的路由感知专家校准机器学习遗忘

    arXiv:2606.10338v1 Announce Type: cross Abstract: Machine unlearning is increasingly important for large language models, yet unlearning in Mixture-of-Experts (MoE) architectures remains underexplored. Unlike dense models, MoE architectures employ a router at each layer to assign…

  4. arXiv cs.AI TIER_1 English(EN) · Christoforos N. Spartalis, Theodoros Semertzidis, Petros Daras, Efstratios Gavves ·

    LoTUS:大规模机器学习遗忘与不确定性的一瞥

    arXiv:2503.18314v5 Announce Type: replace-cross Abstract: We present LoTUS, a novel Machine Unlearning (MU) method that eliminates the influence of training samples from pre-trained models, avoiding retraining from scratch. LoTUS smooths the prediction probabilities of the model …

  5. arXiv cs.CL TIER_1 English(EN) · Sai Li ·

    面向混合专家语言模型的路由感知专家校准机器学习遗忘

    Machine unlearning is increasingly important for large language models, yet unlearning in Mixture-of-Experts (MoE) architectures remains underexplored. Unlike dense models, MoE architectures employ a router at each layer to assign each token to a sparse subset of experts. In this…

  6. arXiv cs.AI TIER_1 English(EN) · Jamie Lanyon, Axel Finke, Petros Andreou, Georgina Cosma ·

    关于多个训练种子在评估机器遗忘中的重要性

    arXiv:2510.26714v5 Announce Type: replace-cross Abstract: Machine unlearning aims to remove the influence of certain data points from a trained model without costly retraining. Most practical unlearning algorithms are only approximate and their performance can only be assessed em…

  7. arXiv cs.AI TIER_1 English(EN) · Praveen Bushipaka, Lucia Passaro, Tommaso Cucinotta ·

    标准采样 vs. 模块化采样:可靠 LLM 遗忘的最佳实践

    arXiv:2509.05316v2 Announce Type: replace-cross Abstract: A conventional LLM Unlearning setting consists of two subsets -"forget" and "retain", with the objectives of removing the undesired knowledge from the forget set while preserving the remaining knowledge from the retain. In…

  8. arXiv cs.LG TIER_1 English(EN) · Youssef Allouah, Joshua Kazdan, Rachid Guerraoui, Sanmi Koyejo ·

    分布内与分布外机器学习遗忘的效用与复杂性

    arXiv:2412.09119v3 Announce Type: replace Abstract: Machine unlearning, the process of selectively removing data from trained models, is increasingly crucial for addressing privacy concerns and knowledge gaps post-deployment. Despite this importance, existing approaches are often…

  9. arXiv cs.CL TIER_1 English(EN) · Gizem Y\"uce, Giorgos Nikolaou, Nicolas Flammarion ·

    学习遗忘什么:通过学习到的令牌级重要性改进 LLM 遗忘能力

    arXiv:2606.06320v1 Announce Type: cross Abstract: Machine unlearning aims to remove targeted knowledge from a trained model while preserving its general capabilities. For autoregressive language models, not all tokens in a forget sample are equally relevant to forgetting. Existin…

  10. arXiv cs.AI TIER_1 English(EN) · Nicolas Flammarion ·

    学习遗忘什么:通过学习到的令牌级重要性改进 LLM 遗忘能力

    Machine unlearning aims to remove targeted knowledge from a trained model while preserving its general capabilities. For autoregressive language models, not all tokens in a forget sample are equally relevant to forgetting. Existing approaches either ignore this heterogeneity or r…

  11. arXiv cs.LG TIER_1 English(EN) · Ahmed Mehdi Inane, Vincent Quirion, Gintare Karolina Dziugaite, Ioannis Mitliagkas ·

    利用非对称源进行遗忘:通过公共数据改进遗忘-效用权衡

    arXiv:2605.11170v2 Announce Type: replace Abstract: Noise-based certified machine unlearning currently faces a hard ceiling: the noise magnitude required to certify unlearning typically destroys model utility, particularly for large-scale deletion requests. While leveraging publi…

  12. arXiv cs.CL TIER_1 English(EN) · Anna Borisiuk, Andrey Savchenko, Alexander Panchenko, Elena Tutubalina ·

    遗忘的解剖:事实显著性和模型微调的双重影响

    arXiv:2602.19612v5 Announce Type: replace Abstract: Machine Unlearning (MU) enables Large Language Models (LLMs) to remove unsafe or outdated information. However, existing work assumes that all facts are equally forgettable and largely ignores whether the forgotten knowledge ori…

  13. arXiv cs.LG TIER_1 English(EN) · Yiwei Chen, Soumyadeep Pal, Yimeng Zhang, Qing Qu, Sijia Liu ·

    模型“遗忘”并非隐形:从模型输出来检测大型语言模型中的“遗忘”痕迹

    arXiv:2506.14003v5 Announce Type: replace Abstract: Machine unlearning (MU) for large language models (LLMs), commonly referred to as LLM unlearning, seeks to remove specific undesirable data or knowledge from a trained model, while maintaining its performance on standard tasks. …

  14. arXiv cs.AI TIER_1 English(EN) · SeungBum Ha, Saerom Park, Sung Whan Yoon ·

    “遗忘”的盲点:过度遗忘与原型再学习攻击

    arXiv:2506.01318v4 Announce Type: replace-cross Abstract: Machine unlearning (MU) aims to expunge a designated forget set from a trained model without costly retraining, yet the existing techniques overlook two critical blind spots: "over-unlearning" that deteriorates retained da…