Researchers have developed a new mixed-precision training framework for Neural Ordinary Differential Equations (Neural ODEs) to reduce computational costs. This framework uses low-precision computations for evaluating network outputs and storing intermediate states, while maintaining numerical stability through custom scaling and high-precision accumulation of solutions and gradients. The approach, accompanied by an open-source PyTorch package named "rampde", achieves approximately 50% memory reduction and up to a 2x speedup in tasks like image classification and generative modeling, with accuracy comparable to single-precision training. AI
影响 Introduces a method to significantly reduce memory and speed up training for Neural ODEs, potentially enabling larger and more complex continuous-time models.
排序理由 This is a research paper detailing a new training method for a specific type of neural network architecture.
AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →
