研究人员推出了一种新颖的方法SIREN-RoPE，通过将旋转位置嵌入（RoPE）的旋转流形视为一个可学习的、信号条件化的空间来增强Transformer架构。该方法通过捕获时间、位置和上下文之间关系的动态组件来增强token的语义含义。在大规模新闻信息流数据集上的评估表明，在计算开销极小的情况下，校准和排名目标得到了一致的改进。 AI
影响 通过引入可学习的旋转空间来增强Transformer中的序列建模，有望改进推荐系统和其他序列感知AI应用。
排序理由 这是一篇介绍Transformer中序列建模新方法的学术论文。
AI 生成摘要 · Google Gemini · 来自 3 个来源。 我们如何撰写摘要 →
arXiv:2604.24717v1 Announce Type: new Abstract: Every Transformer architecture dedicates enormous capacity to learning rich representations in semantic embedding space -- yet the rotation manifold acted upon by Rotary Positional Embeddings (RoPE) has been treated as a fixed, hand…
Every Transformer architecture dedicates enormous capacity to learning rich representations in semantic embedding space -- yet the rotation manifold acted upon by Rotary Positional Embeddings (RoPE) has been treated as a fixed, hand-crafted structure, populated only by discrete o…
Every Transformer architecture dedicates enormous capacity to learning rich representations in semantic embedding space -- yet the rotation manifold acted upon by Rotary Positional Embeddings (RoPE) has been treated as a fixed, hand-crafted structure, populated only by discrete o…