PulseAugur
中
实时 13:37:15
English(EN) Poster: A Preliminary Study of LLM Distillation Inference

新的蒸馏方法旨在提高LLM的能力和安全性

研究人员正在探索用于大型语言模型的高级蒸馏技术,重点是提高能力同时降低风险。一种名为ReCal的方法通过校准剪枝来保留关键的教师支持的预测,从而提高策略内蒸馏恢复能力,带来更好的数学推理和代码生成。另一项研究引入了“用于归罪的蒸馏”(DFI)和“用于能力的蒸馏”(DFC),以揭示或隐藏教师模型的失准,其中DFI在检测隐藏行为方面显示出潜力。此外,SAPD和PIVOT等方法旨在通过优化知识在模型之间的转移方式来改进策略外学习和少样本分类,从而提供计算效率和在特定任务上更好的性能。 AI

影响 这些蒸馏技术的进步可能带来更高效、更安全的LLM开发,从而提高模型性能并降低与失准相关的风险。

排序理由 arXiv上发表了多篇研究论文,详细介绍了与LLM蒸馏相关的新技术和研究。

在 arXiv cs.LG 阅读 →

AI 生成摘要 · Google Gemini · 来自 8 个来源。 我们如何撰写摘要 →

新的蒸馏方法旨在提高LLM的能力和安全性

本文如何被排名

Signal score
0 / 100
Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket
Research
arXiv上发表了多篇研究论文,详细介绍了与LLM蒸馏相关的新技术和研究。
Source corroboration
8 independent sources
Strong cross-source corroboration — multiple independent publishers covered this within the clustering window.
Topics
paper, model release
Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance
High
Clearly on-topic for AI-industry coverage.
Story freshness
53 days old
Aged out of breaking-news scoring windows; ranking reflects the durable signal from the full source set.

完整方法见我们的编辑标准。

报道来源 [8]

  1. arXiv cs.CL TIER_1 English(EN) · Houcheng Jiang, Mao Zheng, Mingyang Song, Qiyong Zhong, Jie Sun, Tianyu Zhang, Junfeng Fang ·

    ReCal: 为在线策略蒸馏恢复校准结构化剪枝

    arXiv:2610.11332v1 Announce Type: new Abstract: Structured pruning reduces the deployment cost of reasoning language models, but the resulting capability degradation can hinder subsequent on-policy distillation (OPD) recovery. Because OPD relies on student-generated trajectories,…

  2. arXiv cs.LG TIER_1 English(EN) · Edward Chen, Yuntao Du ·

    海报:LLM蒸馏推理初步研究

    arXiv:2610.12137v1 Announce Type: cross Abstract: Unauthorized model distillation, in which a model is trained on the outputs of a proprietary large language model (LLM), is a growing threat to model providers. We study distillation inference: determining whether a suspect model …

  3. arXiv cs.AI TIER_1 English(EN) · Sebastian Prasanna, Jacqueline Tay, Alek Westover ·

    用于定罪的蒸馏和用于能力的蒸馏

    arXiv:2610.11012v1 Announce Type: new Abstract: Powerful misaligned AI models might recognize alignment evaluations and strategically behave well on them, rendering direct audits uninformative. However, distilling such a model into a weaker benign student places the teacher in a …

  4. arXiv cs.AI TIER_1 English(EN) · Heng Li, Yong Zhang, Ning Cheng, Zhigen Li, Yun Zhu, Yanmeng Wang, Shaojun Wang, Jing Xiao ·

    PIVOT:面向垂直领域少样本蒸馏的基于困惑度的KD到RL过渡调度

    arXiv:2610.11167v1 Announce Type: cross Abstract: Vertical-domain few-shot classification remains challenging for small language models, as limited supervision makes it difficult to acquire domain-specific decision knowledge. On-Policy Distillation (OPD) can improve teacher-guide…

  5. arXiv cs.CL TIER_1 English(EN) · Tianle Wang, Jiayu Liu, Ruizhi Zhao, Ning Miao ·

    SAPD:步进对齐的特权蒸馏

    arXiv:2610.09665v1 Announce Type: new Abstract: On-policy post-training can improve large language models by learning from their own trajectories, but requires costly rollout generation. We ask whether fixed demonstrations can support competitive off-policy learning through bette…

  6. arXiv cs.AI TIER_1 English(EN) · Yunbo Long, WenJie Chen, Jiaquan Zhang, Guangya Hao, Zihang Zeng, Pengze Li, Xi Chen ·

    SPECTRUM:用于循环自蒸馏的近端谱调制

    arXiv:2610.07237v1 Announce Type: new Abstract: A model that learns from its own outputs inherits more than their correctness: it inherits which solutions it produces. We formulate Looped Self-Distillation, a self-evolution framework for code generation in which a model repeatedl…

  7. Hugging Face Daily Papers TIER_1 English(EN) ·

    ReSAIL:缓解迭代代理自蒸馏中的崩溃

    Iterative self-distillation enables LLM agents to learn from successive deployments, offering a path toward recursive self-improvement (RSI). Yet our experiments with existing methods reveal a collapse in deployment performance across cycles, while task performance with privilege…

  8. LessWrong (AI tag) TIER_1 English(EN) · sebastian_prasanna ·

    [论文] 用于归罪的蒸馏与用于能力的蒸馏

    <p><b><span style="white-space: pre-wrap;">TL;DR: We previously motivated two complementary ways to use distillation for AI safety; we’re releasing a </span></b><a href="https://arxiv.org/pdf/2610.11012"><b><span style="white-space: pre-wrap;">paper</span></b></a><b><span style="…