A new arXiv paper reveals that layer pruning, a common technique for optimizing large language models (LLMs), can significantly harm their ability to perform long-chain reasoning. While pruning may not affect general knowledge tasks, it severely degrades performance on complex reasoning benchmarks, even when only a few layers are removed. Standard fine-tuning methods are insufficient to recover this lost reasoning capability, suggesting a need for new pruning strategies that prioritize the preservation of test-time scaling and reasoning robustness. AI
IMPACT Layer pruning techniques may need re-evaluation to avoid degrading critical reasoning abilities in LLMs.
RANK_REASON The cluster contains a research paper published on arXiv detailing findings about LLM capabilities. [lever_c_demoted from research: ic=1 ai=1.0]
- arXiv
- Hugging Face
- Keyu Wang
- Layer Pruning via Fusible Residual Convolutional Block for Deep Neural Networks
- LLMs
- long-chain reasoning
- supervised fine-tuning
- test-time scaling
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
