Sanjit Dandapanthula

New research reveals core mechanics of reward hacking in generative models

Researchers have identified a fundamental cause of reward hacking in generative models, specifically within flow and diffusion models. They found that a common approximation used in implementing reward guidance, known a…