A new PyTorch optimizer named Rose has been released under the Apache 2.0 license. Developed by Matthew K., Rose is designed to be stateless, offering significantly lower VRAM usage compared to optimizers like AdamW, with memory overhead comparable to plain SGD. Early benchmarks suggest it achieves fast convergence and excellent generalization, even outperforming AdamW on certain tasks and demonstrating competitive results on OpenAI's parameter-golf challenge. AI
影响 Offers a low-VRAM alternative for model training, potentially enabling larger models on consumer hardware.
排序理由 Release of a new open-source optimizer with benchmark results and code.
AI 生成摘要 · Google Gemini · 来自 1 个来源。 我们如何撰写摘要 →