New CACTUS optimizer enhances LLM pruning by preserving influence functions

Researchers have developed CACTUS, a novel optimizer that uses a curvature-aware objective to improve post-training pruning of large language models. This method aims to preserve the internal gradient alignment between sparse and dense models, a feature often lacking in conventional pruning techniques. CACTUS demonstrates significant speedups over existing methods like LLM-Surgeon and shows improved performance on benchmarks such as BIG-Bench Hard when used for data selection. AI

IMPACT This research could lead to more efficient deployment of large language models by enabling significant pruning without substantial performance degradation.

RANK_REASON The cluster contains a research paper detailing a new method for model pruning. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv stat.ML →

• ALPS
• BIG-Bench Hard
• C4 model
• CACTUS
• Llama 3.2:3b
• LLM-Surgeon
• Qwen3-4B
• WikiText-2

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →