L2 regularization
PulseAugur coverage of L2 regularization — every cluster mentioning L2 regularization across labs, papers, and developer communities, ranked by signal.
2 day(s) with sentiment data
-
New Convex Formulation Enhances Analysis of Logistic Regression Poisoning Attacks
Researchers have developed a new convex-analytic reformulation to analyze poisoning attacks on logistic regression models with L2 regularization. This approach replaces sigmoid relations with a convex optimality-gap ine…
-
New L2 regularization variant enhances continual learning in linear networks
Researchers have developed a new weighted variant of L2 regularization designed to improve continual learning in diagonal linear networks. This method aims to preserve the sparsity bias that L2 regularization promotes i…
-
New theory explains how Transformer training affects token clustering
A new theoretical formulation of Transformer dynamics reveals how the training process can influence token clustering during inference. Researchers found that even a single gradient-descent update can cause token distri…
-
New research tackles catastrophic forgetting in LLMs with novel tuning and regularization methods · 6 sources tracked
Researchers are exploring new methods to address catastrophic forgetting in large language models (LLMs) during continual learning. One approach, Subspace Tuning (SST), restricts weight updates to specific subspaces to …
-
Deep Residual Networks Learn Geodesic Curves in Wasserstein Space
A new arXiv paper proposes that deep residual networks (ResNets) learn the geodesic curve within Wasserstein space during training. The research models ResNet forward propagation using continuity equations, suggesting t…
-
New theory explains grokking in deep neural networks via L2 phase transitions
Researchers have developed a new theory explaining the phenomenon of "grokking" in deep neural networks, where a model abruptly begins to generalize after a period of overfitting. The study, published on arXiv, proposes…
-
New regularization methods improve neural network performance and complexity control
Researchers have developed novel norm-based regularization techniques for neural networks, aiming to improve predictive performance and complexity control. These methods extend classical ridge and lasso penalties by inc…