Stop Training for the Worst: Progressive Unmasking Accelerates Masked Diffusion Training
Researchers have introduced Progressive UnMAsking (PUMA), a novel method to accelerate the training of Masked Diffusion Models (MDMs). PUMA aligns the masking patterns used during training with those employed during inference, thereby focusing optimization on more relevant masks. This approach has demonstrated a significant speed-up in pretraining, achieving approximately a 2.5x faster training time at the 125 million parameter scale. AI
IMPACT Accelerates training for diffusion models, potentially enabling faster iteration and development of generative AI.