一项新的研究论文探讨了大型语言模型中期训练阶段的最佳数据构成，发现所有领域的中等比例数据（10%-40%）效果最好。该研究使用了Qwen3-8B-Base模型和KOR-Bench数据集，并揭示了即使在随后的对齐阶段，这些特定领域的性能差距依然存在。此外，研究表明，在中期训练中完全省略某个领域会导致准确率下降，尽管这种影响与模型整体漂移部分混合在一起。 AI
影响 中期训练期间的最佳数据构成对于模型性能和对齐至关重要，这表明需要更具原则性的数据选择策略。
排序理由 该集群包含一篇详细介绍LLM训练方法的论文。
在 Hugging Face Daily Papers 阅读 →
AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →
完整方法见我们的编辑标准。
arXiv:2609.09081v1 Announce Type: new Abstract: Mid-training, the stage between pre-training and alignment, is where a model's per-domain data composition is typically set by data availability rather than principled design. We ask what that decision buys, and whether a later alig…
Mid-training, the stage between pre-training and alignment, is where a model's per-domain data composition is typically set by data availability rather than principled design. We ask what that decision buys, and whether a later alignment pass can undo it. In a controlled logical-…