PulseAugur
中
实时 10:18:09
English(EN) HyScale: Scaling Long-Context Training for Learned Sparse Attention

HyScale系统通过稀疏注意力加速长上下文LLM训练

作者 PulseAugur 编辑部 · [1 个来源] ·

研究人员开发了HyScale，一个旨在优化具有长上下文的大型语言模型训练过程的新系统，特别关注学习到的稀疏注意力机制。该系统通过引入HyTopK进行即时选择和选择重放以进行高效重新计算，解决了注意力相关token选择中的计算瓶颈。HyScale已展示出显著的加速效果，与现有系统相比，对于具有百万token序列的GLM-5.1等模型，训练速度最高可达2.07倍。 AI

影响 优化长上下文模型的训练，可能实现更强大、更高效的AI系统。

排序理由 该集群包含一篇详细介绍新AI模型训练系统的研究论文。[lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.AI 阅读 →

AI 生成摘要 · Google Gemini · 来自 1 个来源。 我们如何撰写摘要 →

HyScale系统通过稀疏注意力加速长上下文LLM训练

本文如何被排名

Signal score
13 / 100
Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket
Tool
该集群包含一篇详细介绍新AI模型训练系统的研究论文。[lever_c_demoted from research: ic=1 ai=1.0]
Source corroboration
Single-source cluster
Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics
paper, infra
Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance
High
Clearly on-topic for AI-industry coverage.
Story freshness
Same-day
Cluster formed today. Ranking reflects the current source set at time of score.

完整方法见我们的编辑标准。

报道来源 [1]

  1. arXiv cs.AI TIER_1 English(EN) · Yangsheng Deng, Xinjing Huang, Yulong Wang, Yunzhuo Liu, Jun Hu, Shuyong Tang, Junchi Zeng, Mingji Han, Ray Ying, Key Zhang, Luo Mai ·

    HyScale：为学习到的稀疏注意力扩展长上下文训练

    arXiv:2610.32057v1 Announce Type: new Abstract: We present HyScale, a long-context training system for learned sparse attention. Learned sparse attention is a promising architecture for million-token contexts, widely adopted in production since DeepSeek-V3.2. Selection, however, …

相关实体

相关话题