Discrete Stochastic Localization for Non-autoregressive Generation
Researchers have introduced Discrete Stochastic Localization (DSL), a new continuous-state framework for non-autoregressive generation. This method aims to improve upon existing discrete diffusion models by offering a more flexible representation that is invariant to signal-to-noise ratio. Fine-tuning a pre-trained model with DSL has shown significant improvements in distributional faithfulness on the OpenWebText dataset, even supporting faster sampling with fewer steps. AI
IMPACT Introduces a novel framework that improves distributional faithfulness and sampling efficiency in generative models.