New ThermoDPO method tackles manifold drift in generative model alignment

Researchers have identified "manifold drift" as a key issue in extending preference optimization methods to continuous-time dynamics for generative models. This phenomenon occurs when reward-driven updates in flow matching move terminal samples off the pretrained data manifold. To address this, a new method called ThermoDPO has been proposed, which uses temperature control to anchor pairwise preference optimization on preferred samples. A weighted variant, ThermoDPO-weighted, further improves performance, achieving a StrictScore of 0.899 on a benchmark and significant gains in OCR and other metrics on the SD3.5-M model. AI

IMPACT Introduces a novel technique to improve the alignment and stability of generative models, potentially leading to more reliable and accurate outputs.

RANK_REASON Academic paper detailing a new method for generative model alignment. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• alphaXiv
• arXiv
• CatalyzeX
• DagsHub
• FlowDPO
• Gotit.pub
• Hugging Face
• ScienceCast
• SD3.5-M
• ThermoDPO
• ThermoDPO-weighted

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →