New MoP framework compresses LLMs, boosting efficiency and accuracy

Researchers have developed a new iterative framework called Mixture of Pruners (MoP) designed to compress Large Language Models (LLMs) by reducing their parameter count and accelerating inference. MoP unifies depth and width pruning, selecting the optimal candidate at each iteration to advance the compression path. This method has demonstrated superior accuracy over existing structured pruning techniques on LLaMA-2 and LLaMA-3 models, achieving a 39% reduction in end-to-end latency at 40% compression. The framework has also been successfully applied to the vision-language model LLaVA-1.5, improving its computational efficiency. AI

IMPACT This research offers a novel approach to model compression, potentially leading to more efficient and accessible LLMs for various applications.

RANK_REASON The cluster describes a new method presented in an academic paper on arXiv. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• Bruno Lopes Yamamoto
• Large Language Models (LLMs)
• LLaMA-2
• LLaMA-3
• LLaVA-1.5
• Mixture of Pruners (MoP)

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →