stochastic gradient descent
PulseAugur coverage of stochastic gradient descent — every cluster mentioning stochastic gradient descent across labs, papers, and developer communities, ranked by signal.
- instance of Influence Flower 90%
- instance of CatalyzeX Code Finder for Papers 90%
- instance of Gotit.pub 90%
- instance of alphaXiv 90%
- used by alphaXiv 70%
- instance of ScienceCast 70%
- used by CORE Recommender 70%
- instance of CatalyzeX 70%
- instance of cs.LG 70%
- instance of Deep Neural Networks 70%
- used by partial differential equations 70%
- used by Influence Flower 70%
7 day(s) with sentiment data
-
New IFT method for throttle valve control faces bias challenges
Researchers have developed a new method called Iterative Feedback Tuning (IFT) to optimize model-free control parameters, particularly for automotive electronic throttle valves. This technique uses stochastic gradient d…
-
New theory precisely analyzes SGD stepsizes for convex optimization
This paper presents an exact worst-case analysis for stochastic gradient descent (SGD) in smooth convex optimization, focusing on the expected suboptimality of the final iterate. The research provides a matching objecti…
-
New research details multi-pass SGD learning dynamics in high-dimensional models
A new paper published on arXiv details the learning dynamics of multi-pass Stochastic Gradient Descent (SGD) in high-dimensional, non-convex multi-index models. The research develops an asymptotically exact characteriza…
-
Research paper dissects locality and weight sharing in CNNs
A new research paper explores the fundamental properties of convolutional neural networks (CNNs), specifically locality and weight sharing. By employing a teacher-student model, the study analyzes how these characterist…
-
New theory proves SGD convergence in deep linear networks
Researchers have established new theoretical convergence guarantees for stochastic gradient descent (SGD) in the context of deep linear neural networks. This work extends existing gradient descent results to SGD under r…
-
New Mesoscopic Model Explains Neural Network Learning Dynamics
Researchers have developed a new mesoscopic description for understanding learning dynamics in extensive-width neural networks. This approach characterizes the distribution of overlaps, which are correlations between le…
-
New recipe for data-adaptive stopping boundaries in iterative algorithms
Researchers have developed a general method for creating data-adaptive stopping boundaries for iterative algorithms. This approach simplifies error analysis by focusing on a single requirement: time-uniform concentratio…
-
New transfer learning method achieves minimax-optimal scaling in regression
Researchers have developed a new method for transfer learning in high-dimensional regression, focusing on the performance improvements achievable through pretrained models. Their work establishes a deterministic equival…
-
New criterion optimizes weight initialization for deep neural networks
Researchers have developed a new mathematical criterion for selecting the optimal variance of weight initialization in deep neural networks (DNNs). By analyzing stochastic gradient descent (SGD) through a continuous-tim…
-
New research explores convergence rates and noise handling in Stochastic Gradient Descent variants
Multiple research papers published on arXiv explore advancements in Stochastic Gradient Descent (SGD) and its variants. One paper focuses on the statistical behavior of SGD with constant step sizes, proving convergence …
-
New research details learning-rate schedules for least-squares regression
A new research paper published on arXiv analyzes the asymptotic risk of learning-rate schedules for least-squares regression with discretely distributed features. The study focuses on the last iterate of stochastic grad…
-
New research analyzes feature normalization impact on finite training budgets
This paper explores the impact of feature standardization on machine learning models trained with finite resources. It analyzes how standardizing rare features affects the learning process and the achievable risks, part…
-
New framework decomposes neural learners to reveal predictability
Researchers have developed a novel probabilistic decomposition method for neural learners, aiming to demystify 'black box' models used in classification and regression. This new framework quantifies heterogeneous predic…
-
New research reveals gradient flow can mispredict neural network adaptation with finite-batch SGD
A new paper explores how population gradient flow, a common method for analyzing neural network adaptation, can inaccurately predict the behavior of finite-batch stochastic gradient descent (SGD). The research identifie…
-
New framework quantifies uncertainty in neural network training dynamics
Researchers have developed a framework to analyze the fluctuations in training dynamics of wide neural networks. This work focuses on how these fluctuations, characterized by functional central limit theorems, propagate…
-
Ising vs. QUBO encodings impact Boltzmann machine learning convergence
A new research paper evaluates the performance of Ising and QUBO variable encodings in Boltzmann machine learning. The study found that QUBO encodings can lead to ill-conditioning in the Fisher Information Matrix, resul…
-
SGD breaks scale symmetries in linear autoencoders, favoring large decoder weights
Researchers have identified a phenomenon in undercomplete linear autoencoders where finite-stepsize stochastic gradient descent (SGD) breaks scale symmetries. This process favors large decoder weights on the principal c…
-
New theory analyzes deep learning optimization convergence
Researchers have developed a new theoretical framework to analyze the convergence properties of optimization algorithms used in deep learning. This approach, drawing on ergodic theory, studies the expected time for algo…
-
New research explores advanced gradient descent for operator learning and optimization
Two new research papers explore advanced gradient descent techniques for complex optimization problems. The first paper details stochastic gradient descent (SGD) for learning operators between Hilbert spaces, establishi…
-
New method approximates stochastic gradient descent over probability measures
Researchers have developed a novel approach to approximate stochastic gradient descent (SGD) dynamics over probability measures, specifically within the Wasserstein space P2. By lifting the problem to a linear Hilbert s…