LLM 'Junk DNA' Hypothesis: Small weights vital for difficult tasks, pruning causes irreversible harm

A new paper introduces the "Junk DNA Hypothesis," challenging the common belief that many parameters in large language models are redundant and can be pruned without consequence. The research suggests that small-magnitude weights, often discarded, are crucial for handling difficult downstream tasks. Removing these weights can lead to irreparable knowledge loss and performance degradation, even with subsequent training, and this effect is more pronounced on harder tasks. AI

Summary written by gemini-2.5-flash-lite from 1 source. How we write summaries →

IMPACT Suggests that aggressive pruning of LLM weights may irreversibly harm performance on complex tasks, contrary to prior assumptions.

RANK_REASON Academic paper introducing a new hypothesis about LLM weight pruning.

Read on arXiv cs.AI →

• Junk DNA Hypothesis
• LLMs
• arXiv

• paper
• safety