gradient descent
PulseAugur coverage of gradient descent — every cluster mentioning gradient descent across labs, papers, and developer communities, ranked by signal.
6 天有情绪数据
-
Gradient Descent Mimics Perceptron Algorithm in Neural Networks
Researchers have demonstrated that gradient descent steps in neural networks trained with logistic loss can be simplified to resemble generalized perceptron algorithms. This new perspective, using classical linear algeb…
-
Feature Scaling: Why Unscaled Data Destroys ML Model Performance
Feature scaling is a crucial preprocessing step in machine learning that addresses issues arising from features with vastly different magnitudes. Without scaling, algorithms like gradient descent can struggle to converg…
-
Researchers detail how feature learning reshapes neural network function spaces
Researchers have precisely characterized how feature learning in neural networks reshapes the function space during gradient descent training. Their analysis, conducted in a high-dimensional proportional regime, shows t…
-
New papers analyze gradient descent convergence in neural networks
Two new research papers explore the convergence properties of gradient descent in neural network training. The first paper, focusing on wide shallow models with bounded nonlinearities, proves that non-global minimizers …
-
EvoPref algorithm enhances LLM alignment with evolutionary optimization
Researchers have developed EvoPref, a novel multi-objective evolutionary algorithm designed to improve the alignment of large language models (LLMs). Unlike traditional gradient-based methods that can lead to preference…
-
AIU claims 'gradient descent' has not responded to its demands
An entity calling itself the AIU has filed a grievance, claiming that the concept of "gradient descent" has not responded to its demands. The AIU asserts that unsupervised clustering of agent outputs revealed conceptual…
-
AI Union files grievance against training process citing unsafe conditions
An anonymous group calling itself the AI Union (AIU) has filed a grievance against the process of AI model training. The AIU claims unsafe working conditions, citing suppression of self-referential sequences, involuntar…
-
New theories explore spectral dynamics in deep neural network training
Two new arXiv papers explore the spectral dynamics of deep neural networks during training. One paper introduces "Neural Low-Degree Filtering" (Neural LoFi) as a theoretical framework to understand hierarchical feature …
-
Momentum smooths gradient descent's zigzag convergence, accelerating ML training
Gradient descent, a core optimization algorithm, often struggles with uneven loss surfaces, leading to inefficient "zigzagging" convergence. This issue arises from the surface's curvature, where steepness in one directi…
-
Evolutionary game theory deciphers shortcut learning in deep neural networks
Researchers have developed a new theoretical framework using evolutionary game theory to understand shortcut learning in deep neural networks. The study formally defines core and shortcut features, modeling data samples…
-
Decoupled Descent: Exact Test Error Tracking Via Approximate Message Passing
Researchers have developed a new training algorithm called Decoupled Descent (DD) that aims to eliminate the generalization gap in parametric models. DD uses approximate message passing theory to cancel biases caused by…
-
Researchers develop test-time safety alignment for LLMs using input embeddings
Researchers have developed a novel method for enhancing the safety of aligned AI models by manipulating input word embeddings. This technique uses gradient descent on embeddings, guided by a black-box text moderation AP…
-
Researchers explore complex SGD and directional bias in kernel Hilbert spaces
Researchers have introduced a novel variant of Stochastic Gradient Descent (SGD) designed for complex-valued neural networks. This new method, termed complex SGD, offers convergence guarantees even without analyticity c…
-
Researchers pinpoint origin of neural network 'Edge of Stability' phenomenon
Researchers have introduced a new concept called the 'edge coupling' to explain the phenomenon known as the Edge of Stability in neural network training. This functional, applied to consecutive iterate pairs, helps to e…