A new research paper explores the learning dynamics of diffusion models, focusing on the interplay between prediction targets (clean data, noise, or velocity) and training loss spaces. The study theoretically demonstrates that for a fixed prediction target, altering the loss space is equivalent to timestep reweighting, influencing generation quality. It also establishes a theoretical lower bound on prediction errors imposed by linear bottlenecks, showing that noise and velocity predictions are more susceptible to these constraints than clean-data predictions. Experiments using U-Net and JiT across various datasets like CIFAR-10 and ImageNet-256 validate these predictions, particularly under compression where prediction choice significantly impacts denoising accuracy and sample quality. AI
IMPACT Provides a quantitative understanding of how architectural bottlenecks interact with prediction and loss choices in denoising generative models.
RANK_REASON Academic paper detailing theoretical and experimental analysis of diffusion models. [lever_c_demoted from research: ic=1 ai=1.0]
- alphaXiv
- arXiv
- CatalyzeX
- CIFAR-10
- Connected Papers
- DagsHub
- Gotit.pub
- Hugging Face
- IArxiv
- ImageNet-256
- Imagenette
- Jit
- Litmaps
- ScienceCast
- Scite
- U-Net
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →