STARFISH: faST Accuracy Recovery in pruned networks From Internal State Healing
Researchers have developed a new method called STARFISH to efficiently restore accuracy in neural networks after pruning. Pruning reduces network size for faster inference but often degrades accuracy. STARFISH optimizes the pruned network by aligning its internal states with the original network's representations using a small set of unlabeled examples. This approach significantly outperforms existing methods, particularly under aggressive pruning scenarios, recovering a much higher percentage of the original accuracy. AI
IMPACT Enhances efficiency of pruned models, potentially accelerating deployment of neural networks in resource-constrained environments.