A new research paper explores how linear diffusion models fail to learn data distribution symmetries, a common characteristic in real-world data like images of dogs. The study identifies three primary sources of this symmetry breaking: a non-symmetric initial weight matrix, a noisy learning target, and finite-sample noise. Researchers found that noise in the learning target is the most persistent cause of symmetry breaking, even when large amounts of data are available. The paper utilizes random matrix theory to quantify symmetry error and suggests that smaller symmetries are more challenging for these models to learn. AI
IMPACT Identifies a key limitation in diffusion models, potentially guiding future research towards more robust symmetry learning.
RANK_REASON Academic paper published on arXiv detailing theoretical findings about diffusion models. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
