Understanding Latent Diffusability via Fisher Geometry
Researchers have developed a new framework to analyze latent-space degradation in diffusion models by quantifying latent-space diffusability using the rate of change of the Minimum Mean Squared Error (MMSE). This framework decomposes the MMSE rate into contributions from Fisher Information (FI) and Fisher Information Rate (FIR), revealing that FIR is influenced by the interaction between encoder and data geometries. The analysis identifies four penalties contributing to diffusion degradation: dimensional compression, tangential distortion, and intrinsic curvatures of both the encoder and the data. Theoretical conditions for preserving FIR are derived to ensure stable diffusability, with experiments across various autoencoding architectures validating these bounds. AI
IMPACT Provides a theoretical framework for understanding and potentially improving the stability and performance of diffusion models in latent spaces.