Multi-armed bandits optimize structured pruning in deep neural networks

Researchers have developed a novel structured pruning framework for deep neural networks that utilizes multi-armed bandit (MAB) algorithms to remove entire neurons. This method treats each neuron as an 'arm' in a bandit problem, temporarily masking it to measure the impact on the loss function before updating its removal reward estimate. Evaluations across image, text, and reasoning tasks demonstrated that MAB-based pruning, particularly with UCB1 and Thompson Sampling policies, effectively reduces model size while often outperforming unpruned models and other pruning techniques. AI

IMPACT Introduces a novel, computationally practical method for structured model reduction that can improve performance and efficiency.

RANK_REASON The cluster contains an academic paper detailing a new method for optimizing deep neural networks. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• Deep Neural Networks
• Thompson Sampling

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →