English(EN)Beyond Next-Token Prediction: A Performance Characterization of Diffusion versus Autoregressive Language Models
扩散语言模型：效率、鲁棒性和路由创新
作者PulseAugur 编辑部·[15 个来源]·
近期研究探索了扩散语言模型（DLM）的进展，重点关注提高其效率和鲁棒性。一篇论文将专家选择路由（Expert-Choice Routing）引入作为DLM的优于令牌选择路由（Token-Choice Routing）的替代方案，实现了更好的负载均衡和更快的收敛。另一项研究提出了AURORA-LM，一种连续潜在扩散语言模型，它将表示构建与分布建模分开，在生成和摘要任务上取得了强劲的性能。进一步的研究调查了ODB-dLLM等加速框架，该框架使用自适应长度预测和推测性解码来加速推理，以及令牌级提前停止以减少扩散步数而不牺牲质量。最后，研究还考察了DLM对噪声和对抗性攻击的鲁棒性，强调虽然它们由于随机性而能抵抗某些攻击，但其整体鲁棒性依赖于权重并且需要架构集成。
AI
Large Language Models (LLMs) have achieved state-of-the-art performance on a broad range of Natural Language Processing (NLP) tasks, including document processing and code generation. Autoregressive Language Models (ARMs), which generate tokens sequentially conditioned on all pre…
arXiv cs.CL
TIER_1English(EN)·Xiaocheng Lu, Hualei Zhang, Shuhan Guo, Jie Zhang, Xiaoyi Pang, Jian Liu, Haoxi Li, Bohai Gu, Haoxuan Che, Jingcai Guo, Song Guo·
arXiv:2608.02942v1 Announce Type: new Abstract: Diffusion language models (dLLMs) can predict many tokens in parallel, but accurate generation still requires many iterative denoising steps. Few-step distillation accelerates decoding by compressing multiple teacher steps into a si…
arXiv cs.AI
TIER_1English(EN)·Tong Ling, Hang Lei, Feng Xiao, Changhui Sun, Jiahang Xie, Hao Liu, Lu Liu, Yanlong Du·
arXiv:2608.03769v1 Announce Type: cross Abstract: Masked diffusion language models (MDLMs) enable parallel generation and bidirectional context modeling, but their positional context differs fundamentally from that of autoregressive (AR) models. Whereas AR decoding exposes a cont…
arXiv cs.AI
TIER_1English(EN)·Brian K Chen, Chong Wu, Kenji Kawaguchi·
arXiv:2608.02625v1 Announce Type: cross Abstract: Diffusion language models (DLMs) can revise tokens bidirectionally, but standard decoding procedures often adapt them to left-to-right generation by producing text block by block. We study a simple plug-and-play inference pattern:…
arXiv:2608.03457v1 Announce Type: new Abstract: Diffusion language models (dLLMs) offer an alternative to autoregressive (AR) language modeling, yet the scaling behavior of Mixture-of-Experts (MoE) dLLMs remains poorly understood. We systematically characterize how optimization h…
arXiv cs.CL
TIER_1English(EN)·Shuibai Zhang, Caspian Zhuang, Chihan Cui, Zhihan Yang, Fred Zhangzhi Peng, Yanxin Zhang, Haoyue Bai, Zack Jia, Yang Zhou, Guanhua Chen, Ming Liu·
arXiv:2608.02602v1 Announce Type: new Abstract: Language remains an outlier in generative modeling: while images, video, and audio are increasingly modeled in continuous latent spaces, text generation still relies predominantly on discrete tokens. Existing continuous language mod…
arXiv cs.CL
TIER_1English(EN)·Linye Wei, Wenjue Chen, Pingzhi Tang, Xiaotian Guo, Le Ye, Runsheng Wang, Meng Li·
arXiv:2511.21759v2 Announce Type: replace Abstract: Diffusion-based large language models (dLLMs) have recently gained significant attention for their exceptional performance and inherent potential for parallel decoding. Existing frameworks further enhance its inference efficienc…
arXiv:2602.11133v2 Announce Type: replace-cross Abstract: Diffusion language models generate text through iterative refinement, a process that is often computationally inefficient because many tokens reach stability long before the final denoising step. We introduce a training-fr…
Diffusion language models (dLLMs) offer an alternative to autoregressive (AR) language modeling, yet the scaling behavior of Mixture-of-Experts (MoE) dLLMs remains poorly understood. We systematically characterize how optimization hyperparameters, compute allocation, and architec…
arXiv cs.CL
TIER_1English(EN)·Yaoxuan Dou, Yang Shu·
arXiv:2607.29079v1 Announce Type: new Abstract: Training-free acceleration makes diffusion-based multimodal large language models (dMLLMs) more deployable, but it may silently change generated content. We study this serving-time consistency problem on 300 real images, comparing F…
Language remains an outlier in generative modeling: while images, video, and audio are increasingly modeled in continuous latent spaces, text generation still relies predominantly on discrete tokens. Existing continuous language models either inherit embedding spaces not designed…
arXiv:2607.27386v1 Announce Type: cross Abstract: Diffusion Language Models (DLMs) offer a compelling alternative to autoregressive (AR) generation by enabling bidirectional context and iterative refinement. However, their reliability under natural input noise and adversarial att…
arXiv cs.CL
TIER_1English(EN)·Kai Syun Hou, James Kwok·
arXiv:2607.24306v1 Announce Type: new Abstract: Diffusion language models enable flexible arbitrary-order generation, but existing sampling methods are mostly designed for early masked diffusion models (MDMs). In this work, we study sampling for recent block diffusion language mo…
Diffusion language models enable flexible arbitrary-order generation, but existing sampling methods are mostly designed for early masked diffusion models (MDMs). In this work, we study sampling for recent block diffusion language models (BDLMs). We show empirically and analytical…
