PulseAugur
实时 14:57:56
English(EN) [Paper] SLAI T-Rex: Full-Parameter Post-training of the DeepSeek-V4 Family on Ascend SuperPOD

SLAI T-Rex框架优化Ascend SuperPOD上的DeepSeek-V4模型

研究人员开发了SLAI T-Rex框架,用于在Ascend SuperPOD基础设施上优化万亿参数MoE模型的全参数后训练。该系统实现了34.22%的模型FLOPs利用率(MFU),比基线方法提高了2.93倍,同时保持了训练稳定性。该框架随后用于使用10K样本数据集创建专门的运筹学(OR)模型,所得的DeepSeek-V4-Flash模型在OR任务上优于GPT-5.4-Mini和基础DeepSeek-V4-Flash。 AI

影响 展示了在非GPU硬件上进行高效万亿参数模型训练和专业化的途径,可能实现更复杂的推理能力。

排序理由 该集群描述了一篇研究论文,其中详细介绍了一种用于在特定硬件上训练大型语言模型的新框架和优化技术。

在 r/LocalLLaMA 阅读 →

AI 生成摘要 · Google Gemini · 来自 3 个来源。 我们如何撰写摘要 →

SLAI T-Rex框架优化Ascend SuperPOD上的DeepSeek-V4模型

报道来源 [3]

  1. arXiv cs.AI TIER_1 English(EN) · Dongfang Li, Xiaodong Luo, Ruoyu Sun, Xuhui Chen, Linyuan Qiu, Jian Meng, Zhengxuan Lu, Yiting Wang, Yucheng Xie, Tao Guo, Tianxiang Fang, Jing Li, Sihang Chen, Shihao Hong, Chang Liu, Weihua Dai, Zirong Zeng, Ziwei Zhu, Zhuohan Wang, Zhengjun Yue, Igor … ·

    SLAI T-Rex:在Ascend SuperPOD上对DeepSeek-V4系列进行全参数后训练

    arXiv:2607.20145v1 Announce Type: cross Abstract: Full-parameter post-training of trillion-parameter-scale MoE models introduces substantial system-level challenges for large-scale distributed training, including severe memory pressure, non-overlapped communication overhead, and …

  2. Hugging Face Daily Papers TIER_1 English(EN) ·

    SLAI T-Rex:在 Ascend SuperPOD 上对 DeepSeek-V4 系列进行全参数后训练

    Full-parameter post-training of trillion-parameter-scale MoE models introduces substantial system-level challenges for large-scale distributed training, including severe memory pressure, non-overlapped communication overhead, and inefficient kernel execution. While most large-sca…

  3. r/LocalLLaMA TIER_1 English(EN) · /u/pmttyji ·

    [Paper] SLAI T-Rex: Full-Parameter Post-training of the DeepSeek-V4 Family on Ascend SuperPOD

    <table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1v47kqc/paper_slai_trex_fullparameter_posttraining_of_the/"> <img alt="[Paper] SLAI T-Rex: Full-Parameter Post-training of the DeepSeek-V4 Family on Ascend SuperPOD" src="https://preview.redd.it/3bpuadlysxeh1.…