Learned token routing in transformers adapts computation depth for efficiency

Researchers have developed a new technique called Token-Selective Attention (TSA) for transformer models that allows them to dynamically adjust the computation depth for each token. This method uses a lightweight, learned gate to decide whether to skip residual updates between transformer blocks, making the process end-to-end differentiable with minimal parameter overhead. TSA demonstrated significant savings in token-layer operations, reducing them by 14-23% on character-level language modeling tasks with less than 0.5% quality loss, and showed improved performance compared to early exit methods at similar efficiency levels. AI

影响 Introduces a method to improve computational efficiency in transformers by adaptively routing tokens, potentially leading to faster inference and reduced training costs.

排序理由 This is a research paper detailing a novel technique for transformer architectures. [lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.LG 阅读 →

• arXiv
• enwik8
• Hugging Face
• Tiny-Shakespeare
• Token-Selective Attention
• Transformer

• 论文
• 模型发布

AI 生成摘要 · Google Gemini · 来自 1 个来源。 我们如何撰写摘要 →