Newton-Schulz
PulseAugur coverage of Newton-Schulz — every cluster mentioning Newton-Schulz across labs, papers, and developer communities, ranked by signal.
3 day(s) with sentiment data
-
New unsupervised method adapts PDE foundation models without ground-truth data
Researchers have developed a new unsupervised finetuning framework for partial differential equation (PDE) foundation models, eliminating the need for ground-truth solutions. This method utilizes a physics-based objecti…
-
New training method improves recurrent memory model performance
A new research paper introduces the "Orthogonalized Read," a training technique designed to improve the performance of recurrent memory models. This method, applied during the read phase of multiplicative LSTMs, acts as…
-
New 'Muse' optimizers explore representation geometry for LLMs
Researchers have introduced "Muse," a novel family of optimizers designed for large language models that explores the geometric properties of parameter representations. Unlike standard Muon-style optimizers, Muse's upda…
-
Turbo-Muon optimizer speeds up AI training with new pre-conditioning technique
Researchers have developed Turbo-Muon, a new pre-conditioning procedure designed to speed up Muon, an optimizer known for its strong performance in large-scale AI training. Turbo-Muon enhances the initialization of the …
-
Matrix Orthogonalization Boosts RNN Memory for Long-Horizon Tasks
Researchers have developed a method to improve the memory capabilities of recurrent neural networks (RNNs) by applying matrix orthogonalization during read operations. This technique, inspired by optimizers used in lang…
-
Muon optimizer needs less orthogonalization than previously thought
Researchers have investigated the optimal level of orthogonalization needed for the Muon optimizer, a technique that enhances neural network training by refining momentum updates. Their study utilized a simplified cubic…
-
Researchers propose novel second-order method for Stiefel manifold optimization
Researchers have developed a novel second-order optimization method for the Stiefel manifold that avoids retractions, offering improved efficiency for high-accuracy requirements. This method combines a tangent component…