Who Should Lead Decoding Now? Tracking Reliable Trajectories for Ensembling Masked Diffusion Language Models
Researchers have introduced TIE (Trajectory-based Iterative Ensembling), a novel framework for combining the knowledge of Masked Diffusion Language Models (MDLMs). TIE leverages the observation that successful MDLM generations display stable confidence dynamics, while unreliable trajectories can be improved by incorporating intermediate states from other models. The framework iteratively identifies reliable decoding trajectories and transfers partially denoised sequences between MDLMs based on evolving confidence levels, allowing different models to contribute their strengths at various stages of generation. AI
IMPACT This research offers a novel approach to improving the performance of Masked Diffusion Language Models by effectively combining their outputs.