New R2D inference technique boosts masked diffusion models on logic tasks

Researchers have developed a new inference technique called Random Remask-Denoise (R$^2$D) for masked diffusion models (MDMs). This method reformulates MDM inference as a Markov chain, incorporating a remask-denoise kernel to analyze the generation trajectory. R$^2$D is a test-time, finetuning-free procedure that can be applied to existing models. Additionally, Trajectory Correction Training (TCT) is proposed to improve denoiser training by exposing the model to its own errors. The approach shows significant performance gains on logical tasks, achieving high accuracy on Sudoku and 3-SAT, and also improves performance on language tasks compared to existing iterative correction methods and LLaDA2.1-mini. AI

IMPACT Introduces a novel inference method that significantly improves performance on logical and language tasks for masked diffusion models.

RANK_REASON Research paper detailing a new inference technique for masked diffusion models. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• arXiv
• Hugging Face
• HumanEval
• LLaDA2.1-mini
• Masked Diffusion Models
• Random Remask-Denoise
• Random Remasking Scales Masked Diffusion Inference
• Trajectory Correction Training

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →