研究人员引入了动态短卷积作为一种新的基元，以增强大型语言模型中使用的Transformer架构。这些动态卷积利用输入相关的滤波器，在保持传统卷积的局部性偏差的同时，提高了表达能力。实验表明，在各种参数规模下，与标准Transformer和静态卷积变体相比，性能持续提高，这表明在计算优势和推进基于Transformer的语言模型方面具有潜力。 AI
影响 引入了一种新颖的技术，为基于Transformer的语言模型提供了计算优势和性能提升。
排序理由 该集群包含一篇详细介绍改进Transformer模型新技术的学术论文。
AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →
arXiv:2606.03825v1 Announce Type: cross Abstract: Transformers have become the dominant architecture for large language models, largely due to the scalability and flexibility of attention, feed-forward layers, residual connections, and normalization. This paper introduces dynamic…
Transformers have become the dominant architecture for large language models, largely due to the scalability and flexibility of attention, feed-forward layers, residual connections, and normalization. This paper introduces dynamic short convolutions as an additional neural networ…