Diffusion models show inherent attention mechanisms and improved sampling techniques · 7 sources tracked
ByPulseAugur Editorial·[13 sources]·
Recent research explores advancements in diffusion models, a dominant architecture for image generation. One paper reveals that these models inherently utilize an attention mechanism similar to transformers, suggesting attention is a universal machine learning principle. Another study introduces Control Variate Score Matching (CVSI) to improve sampling efficiency and reduce variance in diffusion models. Additionally, a new method called Advantage Weighted Matching (AWM) aligns reinforcement learning objectives with pre-training in diffusion models, leading to significant speedups in generation quality.
AI
IMPACT
These advancements could lead to more efficient and versatile diffusion models for various applications, from image and language generation to control tasks.
RANK_REASON
Multiple arXiv papers detailing novel research and theoretical advancements in diffusion models and related machine learning techniques.
arXiv:2609.04090v1 Announce Type: new Abstract: Current conditioned generative models heavily rely on score functions for guidance during training. When the generative model is a diffusion process with a singular diffusion coefficient and the underlying (conditional) densities ei…
arXiv:2506.10226v3 Announce Type: replace-cross Abstract: Synthetic data generation is increasingly used in machine learning for training and data augmentation. Yet, current strategies often rely on external foundation models or datasets, whose usage is restricted in many scenari…
Current conditioned generative models heavily rely on score functions for guidance during training. When the generative model is a diffusion process with a singular diffusion coefficient and the underlying (conditional) densities either do not exist or are not smooth, we use caus…
arXiv cs.LG
TIER_1English(EN)·Patrick Reichherzer, Gianluca Gregori, David N. Hosking, Subir Sarkar·
arXiv:2609.01705v1 Announce Type: new Abstract: Stochastic transport describes physical systems in which an initially structured distribution spreads under unresolved forcing, scattering, or heterogeneous media. Useful surrogates for such systems should be probabilistic, time-res…
arXiv:2511.03913v3 Announce Type: replace-cross Abstract: Deep diffusion models have revolutionized image generation by producing high-quality outputs. However, achieving specific objectives with these models often requires costly adaptations such as fine-tuning, which can be res…
arXiv cs.LG
TIER_1English(EN)·Khaled Kahouli, Romuald Elie, Klaus-Robert M\"uller, Quentin Berthet, Oliver T. Unke, Arnaud Doucet·
arXiv:2512.20003v2 Announce Type: replace Abstract: Sampling from unnormalized probability densities is a pervasive challenge across the computational and physical sciences. Diffusion models provide a powerful generative framework for this task, but their success relies on accura…
arXiv:2609.00885v1 Announce Type: new Abstract: Denoising diffusion models are the dominant architecture for image generation, whereas most natural language generation and modeling are primarily handled by well-known transformer architectures employing attention mechanism. Here, …
arXiv:2509.25050v2 Announce Type: replace Abstract: Reinforcement Learning (RL) has emerged as a central paradigm for advancing Large Language Models (LLMs), where both pre-training and RL post-training stages are grounded in the same log-likelihood formulation. In contrast, rece…
Denoising diffusion models are the dominant architecture for image generation, whereas most natural language generation and modeling are primarily handled by well-known transformer architectures employing attention mechanism. Here, we show that diffusion models also inherently us…
arXiv:2206.09012v4 Announce Type: replace Abstract: We consider the problem of inferring high-dimensional data $\mathbf{x}$ in a model that consists of a prior $p(\mathbf{x})$ and an auxiliary differentiable constraint $c(\mathbf{x},\mathbf{y})$ on $x$ given some additional infor…
arXiv cs.LG
TIER_1English(EN)·Siddarth Venkatraman, Moksh Jain, Luca Scimeca, Minsu Kim, Marcin Sendera, Mohsin Hasan, Luke Rowe, Sarthak Mittal, Pablo Lemos, Emmanuel Bengio, Alexandre Adam, Jarrid Rector-Brooks, Yoshua Bengio, Glen Berseth, Esmeralda S. Whitammer·
arXiv:2405.20971v3 Announce Type: replace Abstract: Diffusion models have emerged as effective distribution estimators in vision, language, and reinforcement learning, but their use as priors in downstream tasks poses an intractable posterior inference problem. This paper studies…
arXiv:2609.01166v1 Announce Type: cross Abstract: The present research is devoted to the nonparametric estimation of a density-dependent drift coefficient in a multivariate McKean--Vlasov diffusion from independent observations at a common time, as well as the stationary density.…
