New method retrofits linear attention to speed up diffusion language models

Researchers have developed a method to retrofit linear attention into diffusion language models (dLLMs) to accelerate inference. This new approach, called block-hybrid attention, combines exact softmax attention within the active denoising block with linear attention over previous blocks. When applied to the LLaDA~2.1 model, resulting in LLaDA-Hybrid, it achieved up to 1.7x higher decoding throughput and improved memory efficiency without significantly compromising performance on benchmarks like HumanEval and MBPP+. AI

IMPACT Accelerates inference for diffusion language models, potentially enabling faster and more efficient deployment of these AI systems.

RANK_REASON The cluster contains an academic paper detailing a new method for improving AI model inference speed. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• arXiv
• Hugging Face
• HumanEval
• LLaDA~2.1
• LLaDA-Hybrid
• MBPP+
• Triton

• paper
• infra

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →