lottery ticket hypothesis
PulseAugur coverage of lottery ticket hypothesis — every cluster mentioning lottery ticket hypothesis across labs, papers, and developer communities, ranked by signal.
1 day(s) with sentiment data
-
New pruning method creates sparse neural networks in one training cycle
Researchers have developed a new method for creating sparse neural networks in a single training cycle, a significant improvement over existing techniques that require multiple cycles. This progressive magnitude-based p…
-
Toy models reveal lottery tickets preserve feature-space geometry
Researchers explored the lottery ticket hypothesis, which suggests that sparse subnetworks within dense neural networks can achieve similar performance to the full model. They used a simplified toy model with a structur…
-
New research links optimizer choice to reduced forgetting in LLM finetuning
Researchers have explored the impact of optimizer consistency during the fine-tuning of large language models. One study suggests that using the same optimizer for both pre-training and fine-tuning leads to less knowled…