Why your diffusion model is slow at batch size 1 (and what actually helps)
Single-image diffusion model inference is slowed by kernel launch overhead and attention memory traffic, rather than raw computational power. Optimizing with `torch.compile` in `reduce-overhead` mode, employing a fused attention backend, and batching classifier-free guidance can significantly reduce latency. Only after these optimizations should one consider distillation methods for further speed improvements, while carefully evaluating potential quality degradation. AI
IMPACT Optimizing diffusion model inference speed can lower operational costs and enable new real-time applications.