新理论优化预训练-微调AI模型的计算分配

作者 PulseAugur 编辑部 · [1 个来源] · 2026-09-16 04:00

研究人员从理论上分析了预训练和微调大型模型的计算分配问题。使用梯度下降训练的正则化最小二乘法作为易于处理的模型，他们描述了在预训练和微调之间分配固定训练预算的最优划分。最优分配取决于预训练方向如何影响微调预测，以及微调的偏移如何通过下游数据几何被感知，特别是与经验协方差的预测相关谱分量相关。 AI

影响为优化预训练和微调中的计算分配提供了一个理论框架，有望带来更高效的模型开发。

排序理由详细介绍AI模型训练理论分析的学术论文。[lever_c_demoted from research: ic=1 ai=1.0]

AI 生成摘要 · Google Gemini · 来自 1 个来源。我们如何撰写摘要 →

本文如何被排名

Signal score: 57 / 100; Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket: Tool; 详细介绍AI模型训练理论分析的学术论文。[lever_c_demoted from research: ic=1 ai=1.0]
Source corroboration: Single-source cluster; Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics: paper, infra; Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance: High; Clearly on-topic for AI-industry coverage.
Story freshness: Breaking (< 6h); Fresh story with cross-source coverage still developing. Ranking may shift as more sources report.

完整方法见我们的编辑标准。

arXiv stat.ML TIER_1 English(EN) · Alex Buna, Fanghui Liu, Patrick Rebeschini · 2026-09-16 04:00

Ridge Gradient Descent 中的计算最优预训练-微调

arXiv:2609.16262v1 Announce Type: new Abstract: Pretraining followed by fine-tuning introduces a compute-allocation problem: under a fixed training budget, compute spent improving the upstream objective reduces the compute available for downstream adaptation. Despite its practica…