Maximal Update Parametrization
PulseAugur coverage of Maximal Update Parametrization — every cluster mentioning Maximal Update Parametrization across labs, papers, and developer communities, ranked by signal.
2 day(s) with sentiment data
-
New DP-μP framework enables hyperparameter transfer for private model training
Researchers have developed DP-μP, a new framework that extends Maximal Update Parametrization (μP) to differentially private optimization. Standard μP, which allows for zero-shot hyperparameter transfer in normal traini…
-
$\\sigma$Transfer enables efficient uncertainty transfer from small to large neural networks
Researchers have developed a method called $\sigma$Transfer that allows for the efficient transfer of predictive uncertainty information from smaller neural networks to significantly larger ones. This technique leverage…
-
New framework predicts hyperparameter transfer laws for neural networks
Researchers have developed a new framework called Hyperparameter Transfer Laws to better understand and predict how hyperparameters should be adjusted when scaling neural network architectures. This framework introduces…
-
Gated Delta Networks scaling rules improve LLM training stability
Researchers have developed new scaling rules for Gated Delta Networks, a type of neural network architecture. These rules, derived through a method called coordinate-size estimation propagation, allow for stable learnin…
-
Probabilistic Transformer scales to 0.4B parameters, outperforming standard models
Researchers have developed a method to scale Probabilistic Transformers (PTs) by transferring hyperparameters from smaller models to larger ones using Maximal Update Parametrization (muP). This technique addresses PT's …