PulseAugur
中
实时 00:10:39
English(EN) RegMix-D: Dynamic Data Mixing via Proxy Training Trajectories

RegMix-D 通过动态数据混合推进 LLM 预训练

作者 PulseAugur 编辑部 · [3 个来源] ·

研究人员推出了 RegMix-D，这是 RegMix 方法在大型语言模型预训练中选择数据混合方面的一项改进。RegMix-D 利用代理运行的完整损失轨迹，而不是仅仅依赖端点损失，来在整个训练过程中动态调整数据混合。这种方法可以离线或在线运行，在 13 项下游任务中，即使代理计算预算显著减少，也显示出比 RegMix 和 DoReMi 等现有方法持续的改进。 AI

影响 该方法可以通过优化数据混合选择，从而实现更高效、更有效的 LLM 训练。

排序理由 该集群描述了 arXiv 论文中提出的一种用于改进 LLM 预训练的新方法。

在 arXiv cs.CL 阅读 →

AI 生成摘要 · Google Gemini · 来自 3 个来源。 我们如何撰写摘要 →

RegMix-D 通过动态数据混合推进 LLM 预训练

本文如何被排名

Signal score
0 / 100
Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket
Research
该集群描述了 arXiv 论文中提出的一种用于改进 LLM 预训练的新方法。
Source corroboration
3 independent sources
Multiple independent publishers reporting the same story raises confidence that it's real and newsworthy.
Topics
paper, infra
Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance
High
Clearly on-topic for AI-industry coverage.
Story freshness
121 days old
Aged out of breaking-news scoring windows; ranking reflects the durable signal from the full source set.
Coverage growth since scoring
+1 source(s) since last score
New sources have picked up this story since our last re-score. Score will update on the next scoring pass.

完整方法见我们的编辑标准。

报道来源 [3]

  1. arXiv cs.CL TIER_1 English(EN) · Kaiyan Zhao, Zhongtao Miao, Akiko Aizawa, Yoshimasa Tsuruoka ·

    RegMix-D：通过代理训练轨迹进行动态数据混合

    arXiv:2606.18663v1 Announce Type: new Abstract: Data mixture selection is critical for Large Language Model pretraining. Existing methods such as RegMix select a single static mixture by fitting a regression model on small-scale proxy runs. We propose RegMix-D, a simple extension…

  2. arXiv cs.CL TIER_1 English(EN) · Yoshimasa Tsuruoka ·

    RegMix-D：通过代理训练轨迹实现动态数据混合

    Data mixture selection is critical for Large Language Model pretraining. Existing methods such as RegMix select a single static mixture by fitting a regression model on small-scale proxy runs. We propose RegMix-D, a simple extension of RegMix to dynamic mixing. Our key observatio…

  3. Hugging Face Daily Papers TIER_1 English(EN) ·

    RegMix-D：通过代理训练轨迹实现动态数据混合

    Data mixture selection is critical for Large Language Model pretraining. Existing methods such as RegMix select a single static mixture by fitting a regression model on small-scale proxy runs. We propose RegMix-D, a simple extension of RegMix to dynamic mixing. Our key observatio…

相关实体

相关话题