New INT8 hardware chip accelerates transformer inference and translation

Researchers have developed the Transformer Accelerator (TFA), a specialized hardware chip designed for efficient INT8 inference of transformer models. This memory-to-memory engine handles both prompt processing and autoregressive generation using a single data path. TFA integrates matrix multiplication, softmax, RMSNorm, and other operations, compiled offline and dispatched via AXI interfaces, supporting various transformer architectures. The design has demonstrated bit-exact execution of pretrained transformers, achieving significant speedups and energy reductions compared to CPU-based inference, and has been successfully synthesized and placed-and-routed on SkyWater sky130. AI

影响 This hardware design could significantly reduce the computational cost and energy consumption for running large transformer models, enabling wider deployment on edge devices.

排序理由 The cluster describes a research paper detailing a new hardware chip design for AI inference. [lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.LG 阅读 →

• INT8
• Machine Translation
• Shashank Chaurasia
• SkyWater sky130
• transformer
• Transformer Accelerator (TFA)

• 基础设施
• 论文

AI 生成摘要 · Google Gemini · 来自 1 个来源。 我们如何撰写摘要 →