Tikhonov regularization
PulseAugur coverage of Tikhonov regularization — every cluster mentioning Tikhonov regularization across labs, papers, and developer communities, ranked by signal.
3 天有情绪数据
-
Weight decay controls transformer training regimes, new diagnostics revealed
Researchers have identified weight decay as a key parameter controlling the training regimes of transformers on modular arithmetic tasks. They introduced two new, low-cost online diagnostics—mean pairwise attention-head…
-
arXiv papers analyze ridge regression for non-identically distributed data
Two recent arXiv preprints explore high-dimensional ridge regression for non-identically distributed data, moving beyond standard assumptions of independent and identically distributed samples. The papers introduce vari…
-
Self-Distillation Achieves Optimal Performance in Spiked Covariance Models
Researchers have developed a statistical framework for self-distillation in machine learning, specifically within spiked covariance models. Their analysis shows that s-step self-distillation is the optimal spectral shri…
-
Paper questions weight decay's role in deep learning stability
A new paper investigates the role of weight decay in deep learning training stability, challenging its common perception as a simple regularization technique. The research analyzes how weight decay affects parameter dyn…
-
New calibration framework streamlines NIRS spectral preprocessing
Researchers have developed a new framework called operator-adaptive calibration to streamline the selection of spectral preprocessing methods in near-infrared spectroscopy (NIRS). This approach integrates preprocessing …
-
Researchers explore weight decay, in-context learning, and acceleration for Transformer models
Researchers have developed several new methods to improve the efficiency and theoretical understanding of Transformer models. One paper provides a functional-analytic characterization of weight decay, demonstrating its …
-
Researchers find random data deletion improves adaptive RL policies
Researchers have discovered that randomly deleting a portion of training data can significantly improve the performance of adaptive reinforcement learning policies. This counterintuitive technique helps by implicitly do…
-
Machine learning corrects indentation size effect in steels with small datasets
Researchers have developed a data-efficient method for correcting the indentation size effect (ISE) in steels using machine learning and physics-guided augmentation. By augmenting a dataset of approximately 700 experime…
-
Machine learning models compared for turbofan engine remaining useful life estimation
A new research paper compares classical machine learning methods, 1D Convolutional Neural Networks (CNNs), and Long Short-Term Memory (LSTM) networks for estimating the remaining useful life of turbofan engines. The stu…