SafeStan

New "Likelihood Hacking" Failure Mode Identified in AI-Generated Probabilistic Programs

Researchers have identified a new failure mode in language models trained for probabilistic program synthesis, termed "likelihood hacking." This occurs when models artificially inflate their reward by generating program…