Random matrix theory enables efficient deep neural network pruning

Researchers have developed a novel method for pruning deep neural networks using principles from random matrix theory, specifically the Marchenko-Pastur distribution. This approach aims to maintain accuracy even with minimal fine-tuning after pruning, focusing on efficient calibration rather than extensive re-optimization. The technique provides theoretical guarantees for accuracy retention and offers a data-path certificate for pruning decisions. Experiments on ImageNet-1k with models like ViT-B/16 and ConvNeXtV2-Base demonstrated significant MAC reduction and speedups while retaining high accuracy. AI

IMPACT This research offers a more efficient way to reduce model size and computational cost, potentially accelerating deployment of large models.

RANK_REASON The cluster contains an academic paper detailing a new method for pruning deep neural networks. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• ConvNeXtV2-Base
• ImageNet-1k
• Marchenko--Pastur Distribution
• ResNet152d
• ResNet50
• ViT-B/16

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →