New Noise Policy Optimization method improves few-step generative models

Researchers have developed a new method called Noise Policy Optimization (NPO) to better align few-step generative models with desired properties, particularly when only black-box reward evaluations are available. Unlike direct reinforcement learning fine-tuning or test-time scaling, NPO learns a policy over the initial noise space of the generator. This approach allows the model to transport Gaussian noise to a reward-tilted distribution, effectively steering the generator's output without altering the pretrained model. NPO demonstrates improved reward-quality trade-offs across image, language, and protein structure generation, achieving comparable results to test-time search methods but with significantly lower inference costs. AI

IMPACT This method could enable more efficient and targeted control over generative models, improving their utility in various applications.

RANK_REASON The cluster contains a research paper detailing a new method for generative models. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• arXiv
• Hugging Face
• Noise Policy Optimization
• Noise-Space Reward Alignment for Few-Step Generative Models
• reinforcement learning

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →