SHIFT-LLM framework corrects depth pruning in LLMs, recovering significant accuracy

Researchers have developed SHIFT-LLM, a novel framework designed to correct accuracy loss in large language models that have undergone depth pruning. This method inserts lightweight Linear Residual Adapters (LRAs) at pruning sites, which approximate the hidden states of the removed blocks without requiring gradient computation. SHIFT-LLM has demonstrated significant accuracy recovery, with gains up to 15.7 points on Llama 3.1 8B-Instruct, using only a few hundred calibration samples. AI

IMPACT This research offers a method to reduce LLM inference costs by pruning layers while mitigating accuracy loss, potentially enabling more efficient deployment of large models.

RANK_REASON Research paper detailing a new method for LLM optimization. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CV →

• alphaXiv
• arXiv
• CatalyzeX
• DagsHub
• Gotit.pub
• Hugging Face
• Linear Residual Adapter
• Llama 3.1 8B-Instruct
• ScienceCast
• SHIFT-LLM

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →