BlockGen: Flexible Blockwise Sequence Modeling with Hybrid Samplers
Researchers have introduced BlockGen, a novel blockwise sequence modeling approach that utilizes hybrid samplers for discrete diffusion. This method explores the effectiveness of uniform-state diffusion models (USDMs) compared to masked diffusion models (MDMs) when generating sequences in blocks rather than token by token. BlockGen integrates autoregressive (AR) predictions with diffusion models to refine unlikely tokens, demonstrating competitive performance on tasks like GSM8K and OpenWebText. AI
IMPACT Introduces a new method for blockwise sequence generation, potentially improving efficiency and performance in discrete diffusion models.