New COEC framework improves LLM pruning accuracy

Researchers have developed a new training-free framework called COEC (Calibrated Orthogonal-Equivalence Compensation) designed to mitigate accuracy degradation in large language models (LLMs) after structured pruning. COEC employs alternating left and right orthogonal rotations to the retained weights, optimizing on a reduced Stiefel manifold and using generalized cross-validation for regularization. Experiments on Llama-3, Llama-3.1, and Qwen2.5 models demonstrate that COEC improves perplexity and zero-shot accuracy compared to existing compensation methods, especially at higher sparsity levels. AI

IMPACT This research offers a method to reduce LLM size and inference costs without significant performance degradation, potentially enabling wider deployment of large models.

RANK_REASON The cluster contains a research paper detailing a new method for LLM pruning. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• Gramian matrix
• Hugging Face
• large language models
• Llama-3
• Llama-3.1
• Qwen2.5
• Stiefel manifold

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →