一篇新的研究论文探讨了预训练课程如何影响Transformer模型的学习和泛化能力。该研究比较了平衡预训练（任务均匀采样）和不平衡预训练（任务顺序引入）。研究结果表明，不平衡课程可以导致神经网络中更解耦的表征，从而提高AI安全应用的微调选择性，例如抑制不当行为。 AI
影响 不平衡预训练课程可能提供一种方法来提高AI系统中安全微调的精度和可靠性。
排序理由 该集群包含一篇在arXiv上发表的研究论文，详细介绍了关于Transformer模型预训练课程的研究结果。
AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →
arXiv:2607.04846v1 Announce Type: cross Abstract: Transformers follow implicit curricula whereby some tasks are learned before others. However, how explicit pretraining curricula influence learning, generalization, and the selectivity of fine-tuning is unclear. This is important …
Transformers follow implicit curricula whereby some tasks are learned before others. However, how explicit pretraining curricula influence learning, generalization, and the selectivity of fine-tuning is unclear. This is important for AI safety, where fine-tuning is used to select…