Brief

Researchers are exploring new methods to improve the efficiency and scalability of diffusion language models (DLMs) for generating long sequences of text. One approach, Block Approximate Sparse Attention (BA-Att), accelerates attention computation by downsampling the attention space, achieving significant speedups while maintaining near full-attention performance. Another development, Dynamic Chunking Diffusion Models (DCDM), replaces fixed positional blocks with content-defined semantic chunks to better capture sequence structure. Additionally, advancements in continuous diffusion models, like RePlaid, demonstrate competitive performance against discrete DLMs, suggesting they are a viable and scalable alternative. AI

Apple's research paper explores the mechanisms behind compositional generalization in conditional diffusion models, specifically focusing on how they handle combinations of conditions not seen during training. The study validates that models exhibiting local conditional scores are better at generalizing, and that enforcing this locality can improve performance. Separately, Hugging Face has released several blog posts detailing various methods for fine-tuning and optimizing Stable Diffusion models, including techniques like DDPO, LoRA, and optimizations for Intel CPUs, as well as instruction-tuning and Japanese language support. AI