English(EN) When Does Blockwise Additivity Break? Diagnosing and Repairing the Additive Assumption in Distillation-Based Architecture Search

研究人员发现架构搜索方法在高压缩率下失效

作者 PulseAugur 编辑部 · [1 个来源] · 2026-11-05 05:46

研究人员在用于架构搜索的块蒸馏方法中发现了一个关键限制，尤其是在DeiT-S和DeiT-B等模型中。这些方法对单个替换块进行加性评分，当压缩率达到原始块延迟预算的55-40%时，这些方法开始显著失效。研究表明，这种失效主要归因于KL加性误差的度量和功能组成部分，其影响因模型架构而异。提出的混合搜索方法通过logit-sum预测器对移动进行排序，并且仅测量最佳候选者，为现有的二次规划方法提供了一种更有效且通常更优越的替代方案。 AI

影响识别当前架构搜索技术的局限性，可能指导开发更鲁棒的模型压缩方法。

排序理由学术论文，详细介绍了关于模型架构搜索方法的新发现。[lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.NE (Neural & Evolutionary) 阅读 →

AI 生成摘要 · Google Gemini · 来自 1 个来源。我们如何撰写摘要 →

本文如何被排名

Signal score: 0 / 100; Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket: Tool; 学术论文，详细介绍了关于模型架构搜索方法的新发现。[lever_c_demoted from research: ic=1 ai=1.0]
Source corroboration: Single-source cluster; Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics: paper, model release; Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance: High; Clearly on-topic for AI-industry coverage.
Story freshness: 5 days old; Aged out of breaking-news scoring windows; ranking reflects the durable signal from the full source set.

完整方法见我们的编辑标准。

报道来源 [1]

arXiv cs.NE (Neural & Evolutionary) TIER_1 English(EN) · Jun Yu · 2026-11-05 05:46

分块可加性何时失效？诊断和修复基于蒸馏的架构搜索中的可加性假设

Blockwise distillation methods for architecture search, such as DNA, DONNA, LANA and Puzzle, score each candidate replacement block in isolation and assemble a child network by minimizing the sum of per-block scores. We test this additive assumption on DeiT-S and DeiT-B with seve…

本文如何被排名

报道来源 [1]

分块可加性何时失效？诊断和修复基于蒸馏的架构搜索中的可加性假设

相关实体

相关话题