PulseAugur
中
实时 06:22:16
English(EN) xHC: Expanded Hyper-Connections

xHC 方法将 Transformer 流扩展到 N=4 以上，以改进 LLM 预训练

作者 PulseAugur 编辑部 · [5 个来源] ·

研究人员推出了一种新颖的 xHC（扩展超连接）方法，用于将 Transformer 模型扩展到 N=4 流的典型限制之外。这种新方法解决了先前超连接（HC）方法中的瓶颈，例如写回信息不足和计算成本高昂的残差混合生成。通过结合时间特征增强和稀疏残差流架构，xHC 能够有效地扩展到 N=16 流，在 18B 和 28B MoE 模型上显示出显著的下游改进。此外，还提出了 xHC-Flash 以减少内存流量，使大 N 残差流扩展在 LLM 预训练中变得实用。 AI

影响 这项研究通过实现 Transformer 架构的更好扩展，有望带来更高效、更强大的大型语言模型。

排序理由 该集群描述了一篇详细介绍改进 Transformer 架构新方法的最新研究论文。

在 arXiv cs.LG 阅读 →

AI 生成摘要 · Google Gemini · 来自 5 个来源。 我们如何撰写摘要 →

xHC 方法将 Transformer 流扩展到 N=4 以上，以改进 LLM 预训练

本文如何被排名

Signal score
0 / 100
Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket
Research
该集群描述了一篇详细介绍改进 Transformer 架构新方法的最新研究论文。
Source corroboration
5 independent sources
Strong cross-source corroboration — multiple independent publishers covered this within the clustering window.
Topics
paper, model release
Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance
High
Clearly on-topic for AI-industry coverage.
Story freshness
55 days old
Aged out of breaking-news scoring windows; ranking reflects the durable signal from the full source set.
Coverage growth since scoring
+2 source(s) since last score
New sources have picked up this story since our last re-score. Score will update on the next scoring pass.

完整方法见我们的编辑标准。

报道来源 [5]

  1. arXiv cs.CL TIER_1 English(EN) · Xiangdong Zhang, Xiaohan Qin, Sunan Zou, Tuo Dai, Xiaoming Shi, Huaijin Wu, Yebin Yang, Zhuo Xia, Shaofeng Zhang, Lin Yao, Yuliang Liu, Yu Cheng, Junchi Yan ·

    xHC: 扩展超连接

    arXiv:2607.14530v1 Announce Type: cross Abstract: Hyper-Connections (HC) expand the residual stream of Transformers into $N$ parallel streams, providing a form of memory scaling beyond model width and depth. Manifold-Constrained HC (mHC) stabilizes this formulation at scale. The …

  2. arXiv cs.LG TIER_1 English(EN) · Junchi Yan ·

    xHC: 扩展超连接

    Hyper-Connections (HC) expand the residual stream of Transformers into $N$ parallel streams, providing a form of memory scaling beyond model width and depth. Manifold-Constrained HC (mHC) stabilizes this formulation at scale. The large gains from $N{=}1$ to $N{=}4$ suggest residu…

  3. Hugging Face Daily Papers TIER_1 English(EN) ·

    xHC: 扩展超连接

    Hyper-Connections (HC) expand the residual stream of Transformers into N parallel streams, providing a form of memory scaling beyond model width and depth. Manifold-Constrained HC (mHC) stabilizes this formulation at scale. The large gains from N{=}1 to N{=}4 suggest residual-str…

  4. r/LocalLLaMA TIER_1 English(EN) · /u/pmttyji ·

    [论文] xHC: 扩展超连接 - 拓宽残差流 · 推动模型智能更进一步

    <table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1v1evsq/paper_xhc_expanded_hyperconnections_scale/"> <img alt="[Paper] xHC: Expanded Hyper-Connections - Scale Residual Streams Wider · Push Model Intelligence Further" src="https://preview.redd.it/trj39nmnace…

  5. dev.to — LLM tag TIER_1 English(EN) · Fardin Sabid ·

    Wormhole Hyperconnections (WHC) v1.0.0

    <p><a class="article-body-image-wrapper" href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fkm4rni1i1nafo569wqcf.png"><img alt=" " height="477" …

相关实体

相关话题