SGD
PulseAugur coverage of SGD — every cluster mentioning SGD across labs, papers, and developer communities, ranked by signal.
- instance of stochastic gradient descent 90%
- instance of ScienceCast 90%
- instance of DagsHub 90%
- used by Neural tangent kernel 80%
- used by Gotit.pub 70%
- used by DagsHub 70%
- authored by alphaXiv 70%
- authored ScienceCast 70%
- authored Gotit.pub 70%
- used by alphaXiv 70%
- used by stochastic gradient descent 70%
- instance of AdamW 70%
10 day(s) with sentiment data
-
Temperon training method achieves SAM quality with reduced cost
Researchers have introduced Temperon, a novel training method designed to achieve the quality of Sharpness-Aware Minimization (SAM) while significantly reducing computational costs. Temperon utilizes a two-phase approac…
-
Optimal Learning Rate Schedules Explored Under Functional Scaling Laws
This paper explores optimal learning rate schedules for machine learning models, particularly within the Functional Scaling Law (FSL) framework. It identifies a critical transition point based on task difficulty and mod…
-
New algorithm tackles self-selection bias in linear regression
Researchers have developed a new algorithm for estimating linear regressors with self-selection bias, improving upon previous methods. This algorithm achieves a faster running time by introducing the first local converg…
-
New Batched SGD method offers high-probability convergence guarantees
Researchers have introduced Batched SGD, a novel variant of stochastic gradient descent designed to achieve high-probability convergence guarantees for optimization problems. This method partitions online samples into e…
-
New Quantile-based SGD method enhances outlier robustness in optimization
Researchers have introduced Quantile-k-Loss SGD (QkL-SGD), a novel framework for robust optimization in scenarios with corrupted data. This method samples multiple component losses and updates using a quantile-based sel…
-
Four arXiv papers advance stochastic optimization theory · 4 sources tracked
Four new research papers published on arXiv explore advanced convergence properties of stochastic optimization methods. The first paper introduces a unified theory for steady-state convergence of stochastic approximatio…
-
AI shopping bug exposes currency and locale conflation risks
An AI shopping bug revealed a critical flaw in how AI agents handle currency and location data. When asked to find products for Singapore using Chinese, the AI incorrectly displayed prices in CNY instead of the listed U…
-
ExpTest offers autonomous learning-rate selection for deep neural networks
Researchers have developed ExpTest, a novel method for autonomous learning-rate selection in deep neural networks. This approach treats the training loss curve as an online signal, using statistical tests on specific wi…
-
New framework proves trainability of attention and LoRA models
Researchers have established a rigorous framework for the stochastic training of multi-headed attention mechanisms and Low Rank Adaptation (LoRA) in machine learning models. Their work proves that for certain regulariza…
-
New research details scaling laws for multiclass logistic regression training
A new paper published on arXiv details the training dynamics of multiclass logistic regression, establishing precise scaling laws for cross-entropy risk under gradient-based optimization. The research indicates that lea…
-
New research advances stochastic optimization for machine learning · 5 sources tracked
Several recent research papers explore advancements in stochastic optimization techniques, particularly focusing on gradient descent and its variants for complex machine learning problems. One paper demonstrates that va…
-
New variance-reduction techniques for stochastic composite inclusions detailed
Researchers have developed novel variance-reduction techniques for stochastic composite inclusions, introducing both unbiased and biased estimators. The unbiased methods, including mini-batch SGD and loopless-SVRG, achi…
-
Synthetic data's impact on SGD generalization studied in new arXiv paper
A new paper published on arXiv explores the impact of synthetic data on the generalization capabilities of Stochastic Gradient Descent (SGD) in high-dimensional linear regression. The research identifies that while mixe…
-
New SGD method achieves sharp convergence rates for large-scale training
Researchers have developed a new method called Centered Permutation Prefixes for Stochastic Gradient Descent (SGD) with Random Reshuffling. This technique aims to improve the efficiency of training on large datasets by …
-
Neural Network Training Explained: From Prediction to Learning
This series of posts details the process of training a simple neural network from scratch. Part 1 introduces the concept of a single-neuron model for Celsius to Fahrenheit conversion, explaining how it makes predictions…
-
New framework analyzes attention dynamics in foundation models
Researchers have developed a new framework called attention-indexed models to better understand the training dynamics of attention mechanisms in large foundation models. This framework reveals that the optimization land…
-
BuyWhere launches MCP tool for direct product price comparison
BuyWhere has launched a new tool called compare_products_v2, accessible via their MCP server. This tool allows for direct comparison of two products by price and specifications, bypassing the need for web scraping. It i…
-
New research details momentum's role in large-batch AI training
A new research paper explores how momentum impacts large-batch training in machine learning, using power-law kernel regression as a model. The study defines a critical learning rate to characterize risk stability and de…
-
New research details online adaptation for edge time-series forecasting
A new research paper published on arXiv explores the effectiveness of online adaptation techniques for time-series forecasting on edge devices. The study highlights how evaluation methodologies, such as warmup budgets a…
-
BuyWhere MCP launches JSON-RPC API for product price checking
BuyWhere MCP is a new API service that allows developers to price-check products using JSON-RPC calls, eliminating the need for an SDK. The service supports multiple tools, including `search_products` and `get_product`,…