$\mu$P
PulseAugur coverage of $\mu$P — every cluster mentioning $\mu$P across labs, papers, and developer communities, ranked by signal.
-
New research reveals learning rate transfer in hybrid Transformer-SSM models
A new research paper explores learning rate (LR) scaling for hybrid architectures that combine Transformer and State-Space Model (SSM) blocks, which are increasingly used in production language models. The study found t…
-
New Graph Metanetworks Enable Cross-Architecture Model Transformations
Researchers have developed Graph Metanetworks (GMNs) to operate directly on the parameters of neural networks, enabling tasks like predicting model properties and transforming trained models. The CrossGMN model specific…
-
Complete-muE framework optimizes hyperparameter transfer for MoE models
Researchers have introduced Complete-muE, a novel framework designed to optimize hyperparameter transfer for Mixture-of-Experts (MoE) models. This system addresses the limitations of existing tools by enabling effective…