Tikhonov regularization
PulseAugur coverage of Tikhonov regularization — every cluster mentioning Tikhonov regularization across labs, papers, and developer communities, ranked by signal.
7 day(s) with sentiment data
-
Linear models with optimized preprocessing match advanced architectures in time-series forecasting
Researchers propose that optimizing preprocessing, rather than scaling model architectures, can significantly improve time-series forecasting accuracy. Using Ridge regression as a testbed, they found that optimal lookba…
-
DREG regularization method shows superior accuracy in deep learning
Researchers have introduced DREG, a layer-wise Jacobian regularization technique that functions as a general-purpose penalty for neural networks. In a large-scale empirical study, DREG demonstrated superior accuracy com…
-
New DSD regularization technique improves ill-conditioned kernel methods
Researchers have developed a new regularization technique called Differential Spectral Damping (DSD) to address ill-conditioned kernel methods, particularly Least-Squares Twin Support Vector Machines (LSTSVM). DSD adapt…
-
New research analyzes Nyström subsampling for domain adaptation
This paper delves into the convergence properties of Nyström subsampling when applied to unsupervised domain adaptation under covariate shift, specifically examining the misspecified case where the target function is ou…
-
Machine Learning Accurately Identifies Ship Hydrodynamics
A new study published on arXiv explores the application of supervised machine learning, specifically regularized regression techniques like Ridge regression, for identifying ship hydrodynamic coefficients. The research …
-
New Federated Unlearning Method Achieves Exact Data Removal for AI Models
Researchers have developed a novel method for federated continual unlearning, specifically designed for models with a frozen foundation and a trainable ridge-regression head. This approach allows for the exact removal o…
-
New training methods boost physical reservoir computer performance
Researchers have developed new training principles for physical reservoir computers, focusing on optical phenomena. The study introduces methods like output pruning and regularization to combat overfitting and improve c…
-
Gradient Descent Outperforms Ridge Regression in Linear Models
A new research paper published on arXiv analyzes the performance of gradient descent (GD) compared to ridge regression and online stochastic gradient descent (SGD) in linear regression tasks. The study finds that GD con…
-
New perturbative method boosts NPIV estimation accuracy
Researchers have developed a novel perturbative approach for non-parametric instrumental variable (NPIV) estimation, drawing inspiration from physics perturbation theory. This method enhances standard kernel ridge techn…
-
Researchers explore grokking phenomenon in ridge regression
Three new research papers explore the concept of "grokking" in machine learning, specifically within the context of ridge regression. One paper presents a numerical procedure to find optimal regularization strength, dem…
-
Weight decay controls transformer training regimes, new diagnostics revealed
Researchers have identified weight decay as a key parameter controlling the training regimes of transformers on modular arithmetic tasks. They introduced two new, low-cost online diagnostics—mean pairwise attention-head…
-
arXiv papers analyze ridge regression for non-identically distributed data
Two recent arXiv preprints explore high-dimensional ridge regression for non-identically distributed data, moving beyond standard assumptions of independent and identically distributed samples. The papers introduce vari…
-
Self-Distillation Achieves Optimal Performance in Spiked Covariance Models
Researchers have developed a statistical framework for self-distillation in machine learning, specifically within spiked covariance models. Their analysis shows that s-step self-distillation is the optimal spectral shri…
-
Paper questions weight decay's role in deep learning stability
A new paper investigates the role of weight decay in deep learning training stability, challenging its common perception as a simple regularization technique. The research analyzes how weight decay affects parameter dyn…
-
New OptMuon method enhances stochastic optimization with adaptive momentum
Researchers have introduced OptMuon, a novel adaptive momentum orthogonalization method for stochastic nonconvex optimization that calibrates update magnitudes from observed trajectories. This approach combines Muon-sty…
-
New calibration framework streamlines NIRS spectral preprocessing
Researchers have developed a new framework called operator-adaptive calibration to streamline the selection of spectral preprocessing methods in near-infrared spectroscopy (NIRS). This approach integrates preprocessing …
-
Researchers explore weight decay, in-context learning, and acceleration for Transformer models
Researchers have developed several new methods to improve the efficiency and theoretical understanding of Transformer models. One paper provides a functional-analytic characterization of weight decay, demonstrating its …
-
Researchers find random data deletion improves adaptive RL policies
Researchers have discovered that randomly deleting a portion of training data can significantly improve the performance of adaptive reinforcement learning policies. This counterintuitive technique helps by implicitly do…
-
Machine learning corrects indentation size effect in steels with small datasets
Researchers have developed a data-efficient method for correcting the indentation size effect (ISE) in steels using machine learning and physics-guided augmentation. By augmenting a dataset of approximately 700 experime…
-
Machine learning models compared for turbofan engine remaining useful life estimation
A new research paper compares classical machine learning methods, 1D Convolutional Neural Networks (CNNs), and Long Short-Term Memory (LSTM) networks for estimating the remaining useful life of turbofan engines. The stu…