Researchers have introduced SARATHI, a novel framework designed to improve the structured pruning of large language models (LLMs). This method decouples the neuron selection and weight reconstruction stages, addressing limitations of previous approaches that coupled these processes. SARATHI utilizes a data-free NMF Residual Subspace Probe for neuron scoring and an Adaptive OBS component for weight reconstruction, significantly reducing memory requirements. A Bias Shift Compensation step further enhances performance by correcting activation drift, particularly in post-LayerNorm architectures. Experiments demonstrate SARATHI's effectiveness across various models and sparsity levels, achieving competitive perplexity and zero-shot accuracy. AI
IMPACT This research could lead to more efficient LLM deployment by reducing inference costs through structured pruning.
RANK_REASON The cluster contains a research paper detailing a new method for LLM pruning. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
