Pruning Deep Neural Networks via the Marchenko--Pastur Distribution
Researchers have developed a novel method for pruning deep neural networks using principles from random matrix theory, specifically the Marchenko-Pastur distribution. This approach aims to maintain accuracy even with minimal fine-tuning after pruning, focusing on efficient calibration rather than extensive re-optimization. The technique provides theoretical guarantees for accuracy retention and offers a data-path certificate for pruning decisions. Experiments on ImageNet-1k with models like ViT-B/16 and ConvNeXtV2-Base demonstrated significant MAC reduction and speedups while retaining high accuracy. AI
IMPACT This research offers a more efficient way to reduce model size and computational cost, potentially accelerating deployment of large models.