New framework sparsifies LLMs without full retraining

Researchers have developed an Adaptive Depth Sparse Framework (AdaDSF) to reduce the inference costs of large language models without extensive retraining. This framework leverages the varying importance of different layers in Transformer architectures, identified by the similarity between layer inputs and outputs. AdaDSF dynamically allocates token retention ratios per layer and uses a lightweight router to select informative tokens, while a feature-preserving alignment objective ensures representation consistency. Experiments on models like GPT-NeoX and Qwen2.5 demonstrate that AdaDSF significantly cuts down FLOPs with minimal performance degradation compared to dense models and other sparsity methods. AI

IMPACT Reduces LLM inference costs, potentially enabling wider deployment and faster response times for AI applications.

RANK_REASON The cluster contains an academic paper detailing a new method for LLM inference optimization. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• AdaDSF
• Adaptive Depth Sparse Framework
• arXiv
• D-LLM
• Dlouhá
• GPT-NeoX
• Qwen2.5
• Transformer++

• paper
• infra

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →