lottery ticket hypothesis
PulseAugur coverage of lottery ticket hypothesis — every cluster mentioning lottery ticket hypothesis across labs, papers, and developer communities, ranked by signal.
1 day(s) with sentiment data
-
New method integrates pruning into active learning to find sparse models
Researchers have developed a new method called Improve & Prune (I&P) that integrates magnitude pruning into active learning retraining cycles. This approach aims to discover sparse subnetworks, or "winning tickets," wit…
-
Sparse few-shot language model for Bengali achieves 90% sparsity
Researchers have developed BnBERT-iPET, a novel approach to sparse few-shot language modeling specifically for Bengali. This method utilizes lottery ticket pruning to achieve 90% sparsity, significantly reducing computa…
-
New 'Double-Scoring' Method Enhances Lottery Ticket Extraction
Researchers have introduced "double-scoring," a novel method for reliably extracting strong lottery tickets from large neural networks. This technique enhances the edge-popup approach by optimizing over an enlarged scor…
-
New pruning method creates sparse neural networks in one training cycle
Researchers have developed a new method for creating sparse neural networks in a single training cycle, a significant improvement over existing techniques that require multiple cycles. This progressive magnitude-based p…
-
Toy models reveal lottery tickets preserve feature-space geometry
Researchers explored the lottery ticket hypothesis, which suggests that sparse subnetworks within dense neural networks can achieve similar performance to the full model. They used a simplified toy model with a structur…
-
New research links optimizer choice to reduced forgetting in LLM finetuning
Researchers have explored the impact of optimizer consistency during the fine-tuning of large language models. One study suggests that using the same optimizer for both pre-training and fine-tuning leads to less knowled…