Insertion Based Sequence Generation with Learnable Order Dynamics
Two new research papers explore advancements in insertion language models (ILMs), a method for generating sequences by inserting tokens. The first paper introduces a continuous-time Markov chain framework to derive a diffusion-style denoising objective for ILMs, showing it can be competitive with existing methods while offering more sampling flexibility. The second paper proposes LoFlexMDM, an ILM that learns data-dependent insertion orders, improving generation quality on molecular tasks. AI
IMPACT These papers advance sequence generation techniques, potentially improving performance in areas like molecular design and general language modeling.