PulseAugur
中
实时 11:33:18
English(EN) How to Allocate Your Tokens? Scaling Laws with Training Steps and Batch Size

新的“三项定律”通过考虑批次大小来完善AI模型缩放

作者 PulseAugur 编辑部 · [2 个来源] ·

提出了一种新的缩放定律，称为“三项定律”，该定律考虑了模型大小和训练数据，特别区分了训练步数和批次大小。该定律已通过拟合大量训练运行数据集得到验证，并准确预测了最佳批次大小。研究表明，与以前的方法相比，使用更少的训练运行即可稳健地拟合此三项定律，并且还可以用于推导次优批次大小的缩放定律，这与现有关于临界批次大小的经验观察一致。 AI

影响 完善了对最佳训练参数的理解，可能导致更高效的模型开发。

排序理由 该集群包含一篇详细介绍AI模型新缩放定律的研究论文。[lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv stat.ML 阅读 →

AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →

新的“三项定律”通过考虑批次大小来完善AI模型缩放

报道来源 [2]

  1. arXiv stat.ML TIER_1 English(EN) · Fabian Schaipp ·

    How to Allocate Your Tokens? Scaling Laws with Training Steps and Batch Size

    arXiv:2607.01487v1 Announce Type: cross Abstract: We propose a scaling law that takes into account model size and training data while explicitly splitting the latter into training steps and batch size (called three-term law). Fitting the proposed law on a large set of training ru…

  2. arXiv stat.ML TIER_1 English(EN) · Fabian Schaipp ·

    How to Allocate Your Tokens? Scaling Laws with Training Steps and Batch Size

    We propose a scaling law that takes into account model size and training data while explicitly splitting the latter into training steps and batch size (called three-term law). Fitting the proposed law on a large set of training runs, we find that it correctly recovers the scaling…

相关实体

相关话题