Researchers have introduced "local redundancy," an information-theoretic measure derived from universal compression theory, to quantify neural network plasticity. This new metric aims to improve upon existing measures like effective rank and dead neuron fraction, which have shown poor correlation with performance on new tasks. The proposed method uses the expected squared gradient norm on a synthetic memorization task as a computable lower bound for local redundancy. Experiments indicate that this measure better predicts downstream performance in continual learning scenarios for image classification and time series tasks. AI
IMPACT Introduces a more principled way to measure neural network plasticity, potentially improving continual and transfer learning performance.
RANK_REASON The cluster contains a research paper detailing a new theoretical measure for neural network plasticity.
- arXiv
- dead neuron fraction
- image classification
- squared gradient norm
- time series transfer learning
- universal compression theory
AI-generated summary · Google Gemini · from 2 sources. How we write summaries →
