Diffusion models struggle with discrete data; new methods improve quality

Researchers have identified a key issue with Gaussian diffusion models when applied to discrete data, specifically noting that the DDPM solver struggles with sampling intervals that lead to multimodal distributions. This can result in the generation of out-of-distribution inputs, degrading sample quality. The paper proposes solutions including self-conditioning and a novel q-sampling solver, demonstrating improved generation quality across text, code, and protein domains when these methods are combined. AI

IMPACT Addresses a core limitation in diffusion models, potentially improving their applicability to text and code generation tasks.

RANK_REASON The cluster contains an academic paper detailing a technical problem and proposed solutions for diffusion models. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• Gaussian Diffusion Models
• Ildus Sadrtdinov

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →