English(EN)Understanding and Accelerating the Training of Masked Diffusion Language Models
新研究加速扩散语言模型训练并增强生成能力
作者PulseAugur 编辑部·[6 个来源]·
研究人员正在探索掩码扩散语言模型（MDMs）的进步，以提高其训练效率和生成能力。一项研究提出了一种“钟形时间采样”策略，可将MDM训练速度提高多达四倍，同时在One Billion Word Benchmark等基准测试中保持性能。另一篇论文介绍了多掩码扩散模型（MultiMDMs），通过保留掩码结构来增强少步生成，从而在精炼为干净标记之前更好地预测指定掩码。此外，一个名为基于轨迹的策略内蒸馏（TOPD）的新框架允许扩散语言模型在没有奖励估计的情况下转移推理能力，在数学推理任务上取得了具有竞争力的结果，而计算量却大大减少。最后，一项分析显示，扩散语言模型通过双向电路实现归纳，利用过去和未来的上下文进行上下文学习。
AI
arXiv:2607.24841v1 Announce Type: new Abstract: Autoregressive (AR) large language models (LLMs) are inherently inefficient at inference time because each generated token requires accessing the full set of model parameters, leading to low operational intensity and high energy con…
arXiv cs.AI
TIER_1English(EN)·Chunsan Hong, Sanghyun Lee, Chieh-Hsin Lai, Satoshi Hayakawa, Yuhta Takida, Yuki Mitsufuji, Seungryong Kim, Jong Chul Ye·
arXiv:2605.13026v2 Announce Type: replace-cross Abstract: Masked diffusion models (MDMs) have emerged as a promising alternative to autoregressive models (ARMs) for language modeling. However, MDMs are known to learn substantially more slowly than ARMs, which may become problemat…
arXiv:2607.19686v1 Announce Type: new Abstract: Masked diffusion models (MDMs) are a promising family of language generators, but achieving high-quality few-step generation remains challenging. In MDMs, all forward trajectories collapse to a single fully masked state, leaving no …
arXiv cs.AI
TIER_1English(EN)·Haolin Ren, Ziyang Huang, Chenhao Yuan, Jun Zhao, Kang Liu·
arXiv:2607.16872v1 Announce Type: cross Abstract: Diffusion large language models (dLLMs) are a promising alternative to autoregressive generation. However, reasoning-oriented post-training for dLLMs remains challenging. Supervised fine-tuning (SFT) for dLLMs requires dense but o…
arXiv:2607.15893v1 Announce Type: cross Abstract: While the internal mechanisms of autoregressive (AR) transformers have been studied extensively, much less is known about diffusion language models (DLMs), an emerging alternative that generates text by iterative denoising. In thi…
dev.to — LLM tag
TIER_1English(EN)·Devanshu Biswas·
<p>Almost every LLM you've used — GPT, Claude, Llama — writes text the way a person types: predict the next token, append it, predict the next one from everything so far. One token per forward pass, strictly left-to-right. It's so standard that most people assume it's just <em>wh…
