English(EN) UltraX: Refining Pre-Training Data at Scale with Adaptive Programmatic Editing

UltraX框架通过自适应程序化编辑优化LLM预训练数据

作者 PulseAugur 编辑部 · [2 个来源] · 2026-07-09 16:18

研究人员推出UltraX，一个旨在优化大型语言模型（LLM）大规模预训练数据的新型框架。该系统通过专注于通过自适应程序化编辑来提高数据质量，解决了仅增加数据量带来的收益递减问题。UltraX通过实现细粒度的实例级编辑（包括插入、删除和修改）来提高数据利用率，并采用可靠的程序监督生成流程。实验表明，UltraX提高了数据效率和优化可靠性，与现有方法相比，在训练的token更少的情况下实现了高性能。 AI

影响提高了LLM训练的数据效率和可靠性，有望用更少的数据获得更好的模型性能。

排序理由该集群描述了一篇研究论文，详细介绍了一种优化大型语言模型训练数据的新方法。

在 arXiv cs.AI 阅读 →

AI 生成摘要 · Google Gemini · 来自 2 个来源。我们如何撰写摘要 →

报道来源 [2]

arXiv cs.AI TIER_1 English(EN) · Xinlong Zhao, Dongsheng Liu, Hengyu Zhao, Zixuan Fu, Zheng Wang, Jie Cai, Jie Zhou, Qiang Ma, Xuanhe Zhou, Xu Han, Yudong Wang, Zhiyuan Liu · 2026-07-10 04:00

UltraX: Refining Pre-Training Data at Scale with Adaptive Programmatic Editing

arXiv:2607.08646v1 Announce Type: cross Abstract: As available training data approaches its physical limit, gains from Scaling Laws have begun to diminish. Consequently, improving Large Language Models (LLMs) now depends less on data expansion and more on higher-quality data util…
arXiv cs.AI TIER_1 English(EN) · Zhiyuan Liu · 2026-07-09 16:18

UltraX: Refining Pre-Training Data at Scale with Adaptive Programmatic Editing

As available training data approaches its physical limit, gains from Scaling Laws have begun to diminish. Consequently, improving Large Language Models (LLMs) now depends less on data expansion and more on higher-quality data utilization. However, in the context of large-scale co…

报道来源 [2]

UltraX: Refining Pre-Training Data at Scale with Adaptive Programmatic Editing

UltraX: Refining Pre-Training Data at Scale with Adaptive Programmatic Editing

相关实体

相关话题