研究人员推出了一种新颖的架构Cubit，它用核岭回归（KRR）取代了Transformer中的注意力机制。这种方法在最近的一篇arXiv论文中有详细介绍，与传统的Transformer相比，它提供了更强的数学基础，并可能提高长序列建模能力。另一篇论文将可微分核岭回归（KRR）作为深度学习管道的模块化组件进行探索，证明其能够以更少的训练匹配或增强现有模型。 AI
影响 引入了可能改进长序列建模并提供标准Transformer注意力机制替代方案的新架构组件。
排序理由 该集群包含两篇arXiv论文，详细介绍了用于深度学习架构的核方法的最新研究。
AI 生成摘要 · Google Gemini · 来自 4 个来源。 我们如何撰写摘要 →
完整方法见我们的编辑标准。
arXiv:2605.06501v1 Announce Type: new Abstract: Since its introduction in 2017, the Transformer has become one of the most widely adopted architectures in modern deep learning. Despite extensive efforts to improve positional encoding, attention mechanisms, and feed-forward networ…
Since its introduction in 2017, the Transformer has become one of the most widely adopted architectures in modern deep learning. Despite extensive efforts to improve positional encoding, attention mechanisms, and feed-forward networks, the core token-mixing mechanism in Transform…
arXiv:2605.02313v1 Announce Type: new Abstract: Deep neural networks dominate modern machine learning, while alternative function approximators remain comparatively underexplored at scale. In this work, we revisit kernel methods as drop-in components for standard deep learning pi…
Deep neural networks dominate modern machine learning, while alternative function approximators remain comparatively underexplored at scale. In this work, we revisit kernel methods as drop-in components for standard deep learning pipelines. We introduce \emph{Sparse Kernels} (SKs…