一篇新论文探讨了机器学习模型微调过程中数据洗牌的顺序如何引入显著噪声。这种源于AdamW和SGD等优化器内存的噪声，甚至可能颠覆A/B比较的结果。该研究提出了一种在不拟合参数的情况下量化这种噪声的方法，从而深入了解顺序方差并为微调比较提供标准。 AI
影响 强调了模型训练中一个先前被低估的因素，该因素可能影响可复现性和性能比较。
排序理由 该集群包含一篇详细介绍新研究发现的学术论文。[lever_c_demoted from research: ic=1 ai=1.0]
AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →
arXiv:2606.29554v1 Announce Type: cross Abstract: Shuffle order can be a larger source of fine-tuning noise than a memoryless analysis predicts: fixed-clock optimizer memory makes local equal-multiset contrasts first order in the learning rate rather than second order, and the re…
Shuffle order can be a larger source of fine-tuning noise than a memoryless analysis predicts: fixed-clock optimizer memory makes local equal-multiset contrasts first order in the learning rate rather than second order, and the resulting order channel can be large enough for a si…