研究人员开发了新的方法来提高Transformer模型在多个设备上推理的效率。一种名为ASTRA的方法，将序列并行与混合精度注意力相结合，以减少设备间带宽需求，即使在低带宽网络上也能实现显著的加速。另一个框架Meta-Attention使用贝叶斯元控制器动态地将token路由到最合适的注意力策略，提供了更好的计算-性能权衡。此外，一项关于嵌入式边缘设备的研究表明，驱动剖析的适应对于实际的分布式Transformer推理至关重要，通过降低延迟和能耗，其性能优于静态分布式设置。 AI
影响 这些进步可以显著降低部署大型AI模型的计算成本和延迟，从而在各种硬件上实现更高效的实时应用程序。
排序理由 多篇研究论文详细介绍了高效Transformer推理的新颖方法。
AI 生成摘要 · Google Gemini · 来自 5 个来源。 我们如何撰写摘要 →
arXiv:2505.19342v2 Announce Type: replace-cross Abstract: Multi-device inference can reduce Transformer latency by parallelizing computation. However, existing methods require high inter-device bandwidth, making them impractical for bandwidth-constrained environments. We present …
arXiv:2605.28384v1 Announce Type: new Abstract: Standard transformer architectures apply a single attention mechanism uniformly across all tokens and sequence positions, irrespective of local context or computational budget. We propose Meta-Attention, a framework that dynamically…
Standard transformer architectures apply a single attention mechanism uniformly across all tokens and sequence positions, irrespective of local context or computational budget. We propose Meta-Attention, a framework that dynamically routes each token to the most appropriate atten…
arXiv:2605.25682v1 Announce Type: cross Abstract: Distributing Transformer inference across embedded edge devices can alleviate individual memory and compute constraints, yet practical benefits on real hardware remain unclear: prior work relies largely on simulations that overloo…
Distributing Transformer inference across embedded edge devices can alleviate individual memory and compute constraints, yet practical benefits on real hardware remain unclear: prior work relies largely on simulations that overlook hardware-specific communication overheads. We pr…