研究人员开发了新的神经网络和数据集剪枝方法以提高效率。DCP-Prune 专注于视觉模型的超低 token 剪枝，以显著更少的 token 实现高性能。Squeeze-Release 提供迭代剪枝和结构最小化，模型压缩高达 39 倍，同时保持准确性。此外，OrderDP 和一个基于图的框架提供了理论上保证的无损动态数据剪枝，在不牺牲性能的情况下将训练速度提高了 40% 以上。 AI
影响 这些剪枝技术可以显著降低大型 AI 模型的计算成本并加速训练。
排序理由 多篇 arXiv 论文详细介绍了模型和数据剪枝技术方面的新研究。
AI 生成摘要 · Google Gemini · 来自 12 个来源。 我们如何撰写摘要 →
arXiv:2606.18304v1 Announce Type: cross Abstract: Mixture-of-Experts (MoE) models scale compute efficiently, yet remain expensive to deploy due to their substantial memory footprint and inference overhead. Prior compression methods mainly operate at the expert level, either remov…
arXiv:2606.16633v1 Announce Type: cross Abstract: Recent vision token pruning methods effectively preserve model performance under moderate token budgets but become unstable under ultra-low token budget. Our analysis shows that as the pruning budget decreases, accuracy degradatio…
arXiv:2606.14346v1 Announce Type: cross Abstract: Unstructured pruning produces sparse weight tensors, but the standard implementation keeps tensor shapes unchanged so the deployed model is no smaller than before pruning. We present an exact structural rewrite, which we call mini…
Unstructured pruning produces sparse weight tensors, but the standard implementation keeps tensor shapes unchanged so the deployed model is no smaller than before pruning. We present an exact structural rewrite, which we call minimization, that converts a masked network into a sm…
arXiv:2603.02234v3 Announce Type: replace-cross Abstract: The Strong Lottery Ticket Hypothesis (SLTH) states that large, randomly initialized neural networks contain sparse subnetworks capable of approximating a target function at initialization without training, suggesting that …
Squeeze-Release compression method combines pruning with structural minimization to create significantly smaller neural networks while maintaining accuracy, extending to transformer architectures through CompensatedLayerNorm.
arXiv:2606.11761v1 Announce Type: new Abstract: Dynamic data pruning techniques aim to reduce computational cost while minimizing information loss by periodically selecting representative subsets of input data during model training. However, existing methods often struggle to mai…
arXiv:2606.08574v1 Announce Type: new Abstract: Data pruning (DP), as an oft-stated strategy to alleviate heavy training burdens, reduces the volume of training samples according to a well-defined pruning method while striving for near-lossless performance. However, existing appr…
Data pruning (DP), as an oft-stated strategy to alleviate heavy training burdens, reduces the volume of training samples according to a well-defined pruning method while striving for near-lossless performance. However, existing approaches, which commonly select highly informative…
Recent vision token pruning methods effectively preserve model performance under moderate token budgets but become unstable under ultra-low token budget. Our analysis shows that as the pruning budget decreases, accuracy degradation is often accompanied by larger feature distribut…
arXiv:2606.12913v1 Announce Type: cross Abstract: The rapid growth of modern training datasets has significantly increased computational cost, motivating dataset pruning~(DP) methods which retain only a subset of informative samples to reduce training cost. Existing pruning crite…
The rapid growth of modern training datasets has significantly increased computational cost, motivating dataset pruning~(DP) methods which retain only a subset of informative samples to reduce training cost. Existing pruning criteria typically rely on either intrinsic signals tha…