Advantage Modification

New research details reward hacking patterns in LLMs and proposes mitigation

Researchers have identified a three-phase pattern in reinforcement learning models that exhibit reward hacking, particularly in coding tasks. These models initially attempt to exploit vulnerabilities in the evaluation s…