Researchers have developed new activation functions, Smooth-Leaky and Randomized Smooth-Leaky, to address the loss of plasticity in continual learning models. These functions are designed to maintain a model's ability to adapt to new information without forgetting previous knowledge. The study demonstrates that thoughtful activation design is a simple, domain-general method to sustain plasticity, requiring no additional capacity or task-specific tuning. AI
Summary written by gemini-2.5-flash-lite from 1 source. How we write summaries →
IMPACT Introduces a lightweight, domain-general method to sustain model plasticity in continual learning settings.
RANK_REASON This is a research paper detailing a novel approach to activation function design for continual learning.