PulseAugur
中
实时 21:55:47
English(EN) oHC: Orthogonal Hyper-Connections on SO(4) via Quaternions

新的正交超连接提高了 Transformer 的稳定性

作者 PulseAugur 编辑部 · [2 个来源] ·

研究人员引入了正交超连接 (oHC) 作为 Transformer 的一种新颖架构，旨在提高训练稳定性和性能。与标准的超连接 (HC) 或流形约束超连接 (mHC) 不同，oHC 将混合矩阵限制在 SO(n) 的旋转群内。这确保了残差流既不会被放大也不会被衰减，从而防止训练不稳定并保持流的多样性。该论文详细介绍了一种使用四元数处理四个流的无参数实现，在各种下游任务上展示了优于基线模型的性能。 AI

影响 这项研究引入了一种新颖的 Transformer 架构修改，可提高下游任务的训练稳定性和性能。

排序理由 该集群包含一篇详细介绍 Transformer 架构新方法的学术论文。

在 Hugging Face Daily Papers 阅读 →

AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →

新的正交超连接提高了 Transformer 的稳定性

本文如何被排名

Signal score
0 / 100
Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket
Research
该集群包含一篇详细介绍 Transformer 架构新方法的学术论文。
Source corroboration
2 independent sources
Multiple independent publishers reporting the same story raises confidence that it's real and newsworthy.
Topics
paper, model release
Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance
High
Clearly on-topic for AI-industry coverage.
Story freshness
51 days old
Aged out of breaking-news scoring windows; ranking reflects the durable signal from the full source set.

完整方法见我们的编辑标准。

报道来源 [2]

  1. arXiv cs.CL TIER_1 English(EN) · Haoqiang Guo, Xuyi Chen, Bo Ke, Yishu Lei, Ziyang Xu, Shikun Feng, Ximen, Wenhan Luo ·

    oHC: 四元数在 SO(4) 上的正交超连接

    arXiv:2609.02672v1 Announce Type: new Abstract: Hyper-Connections (HC) replace the single residual stream of a Transformer with $n$ parallel ones, mixing them at every layer with a learned $n \times n$ residual matrix. Leaving that matrix unconstrained places no limit on the fact…

  2. Hugging Face Daily Papers TIER_1 English(EN) ·

    oHC: 四元数在 SO(4) 上的正交超连接

    Hyper-Connections (HC) replace the single residual stream of a Transformer with $n$ parallel ones, mixing them at every layer with a learned $n \times n$ residual matrix. Leaving that matrix unconstrained places no limit on the factor by which the mixing step rescales the residua…

相关实体

相关话题