Diffusion models' score insensitivity and mixture weight recovery analyzed

Researchers have identified a puzzling behavior in score-based generative models where they may fail to learn accurate relative mode amplitudes, also known as mixture weights, despite covering all modes of a target multimodal distribution. A new framework, the Diffusion Score Sensitivity Index (DSSI), has been developed to quantify the variation in the diffusion score matching (DSM) loss relative to parameter changes. This index governs how accurately parameters of the target distribution can be estimated from generated samples, with Gaussian mixtures demonstrating that estimation errors can be on the same order as the DSM loss under mild conditions. The study also found that the choice of noise schedule can influence diffusion sensitivity, potentially leading to mode amplification. AI

IMPACT Provides a theoretical framework and metrics to understand and potentially improve the training of generative models for multimodal distributions.

RANK_REASON Academic paper detailing a new theoretical framework and empirical analysis of diffusion models. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv stat.ML →

• arXiv
• Diffusion Models
• Diffusion Score Matching (DSM)
• Diffusion Score Sensitivity Index (DSSI)
• Gaussian Mixture Models

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →