Researchers have developed Step-dLLM, a novel method to improve the efficiency of diffusion large language models (dLLMs) during inference. Unlike autoregressive models, dLLMs cannot use KV caching and recompute attention maps at each step, leading to high costs. Step-dLLM addresses this by adaptively refreshing sparse attention masks at anchor steps and employing a head-adaptive budget that prioritizes heads with concentrated attention. This approach, implemented with a fused Triton kernel, achieves significant speedups on models like LLaDA-1.5 and Dream-7B-Instruct without retraining, while also improving the accuracy-latency trade-off compared to existing sparse attention methods. AI
IMPACT This new method could significantly reduce the computational cost of running diffusion LLMs, making them more accessible and efficient for various applications.
RANK_REASON Research paper detailing a new method for improving LLM inference efficiency. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
