muon
PulseAugur coverage of muon — every cluster mentioning muon across labs, papers, and developer communities, ranked by signal.
9 天有情绪数据
Spectrum preservation is a common theme in new optimizer research
The introduction of Pion, which 'preserves spectrum', and Muown, which addresses 'spectral norm drift', indicates a broader trend in optimizer development. This focus on maintaining spectral properties suggests that current optimizers, including Muon, may suffer from spectral instability that hinders training.
Muon optimizer's spectral norm drift is a key area for improvement
Multiple recent papers (Muown, Pion, and the general mode connectivity research) highlight issues related to spectral norms and Muon. Muown explicitly addresses 'upward drift of spectral norms', while Pion aims to 'preserve spectrum'. This suggests that managing spectral properties is a critical challenge for Muon's stability and performance.
Aurora optimizer may outperform Muown in addressing Muon's neuron death
Tilde Research's Aurora optimizer is specifically designed to fix 'neuron death' in Muon, a problem not explicitly addressed by Muown. While Muown improves spectral norm drift, Aurora's targeted approach to neuron inactivity could lead to more comprehensive performance gains, especially in scenarios where neuron death is a primary bottleneck.
Muon's spectral properties are being actively studied in relation to optimizer behavior and mode connectivity
Multiple recent clusters highlight research into Muon's spectral properties and how they interact with optimization dynamics. The connection between optimizers, spectral norms, and mode connectivity suggests ongoing theoretical and empirical work is exploring fundamental aspects of Muon's behavior.
Muon's neuron death issue may be addressed by new optimizers like Aurora within 3 months
The Tilde Research launch of Aurora specifically targets neuron death in Muon. Given Aurora's public release and demonstrated effectiveness, it's plausible that Muon users will adopt Aurora or similar solutions to mitigate this issue within the next quarter.
-
像Muon这样的谱优化器在联想记忆任务中表现出急剧的容量缩放
一篇新论文分析了像Muon这样的谱优化器在训练大型语言模型中的性能，通过检查它们在学习联想记忆方面的有效性。研究表明，在存储联想方面，Muon显著优于标准的随机梯度下降（SGD），甚至在使用仅有一阶信息的情况下也能媲美牛顿法。该研究还强调了与SGD相比，Muon的临界批次大小更大，初始恢复率更快，从而对谱预处理器的信号放大进行了量化理解。
-
DeepSeek V4 models offer high performance with reduced inference costs and NPU support
DeepSeek has released its V4 family of open-weight large language models, featuring a 1.6 trillion parameter model and a smaller 284 billion parameter Flash MoE model. These new models claim to rival top proprietary LLM…
-
Anthropic 更新 Claude Code，增强后台会话并修复错误
Anthropic 发布了其开发工具 Claude Code 的多个更新，涵盖版本 v2.1.141 至 v2.1.150。这些更新为后台会话管理、插件功能和工具集成带来了显著改进，尤其针对 Windows 用户。主要增强功能包括更好地处理空闲会话、更可靠的自动更新程序错误报告，以及用于配置后台代理的扩展命令行选项。此次发布还解决了与权限、沙盒和用户界面响应能力相关的多项错误，旨在提供更稳定、更高效的编码环境。