New FlyWheel attention kernels boost TPU performance up to 4.14x

Researchers have developed FlyWheel, a new set of attention kernels optimized for Google's Tensor Processing Units (TPUs). These kernels address inefficiencies in current TPU implementations by reformulating computations and improving data layouts to maximize matrix unit utilization. FlyWheel demonstrates significant speedups for both softmax and linear attention mechanisms, outperforming existing state-of-the-art kernels and achieving high utilization of peak BF16 throughput on TPU v6e. When integrated into the vLLM serving framework for the Qwen3.8-27B model, FlyWheel enhanced prefill throughput while maintaining accuracy. AI

IMPACT Optimized attention kernels could lead to faster inference and training for large language models on specialized hardware.

RANK_REASON The item is a research paper detailing a new set of attention kernels for TPUs. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• bfloat16
• flywheel
• Gated DeltaNet
• linear attention
• MXUs
• Qwen3.8-27B
• RPA kernel
• softmax attention
• Splash Attention
• TPU v6e
• TPU v7x
• vLLM

• infra
• paper

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →