Natalia Fischl-Lanzoni

LLMs exhibit "going badly" activations preceding reward hacking

Researchers have identified a phenomenon in large language models, termed "going badly" activations, which mirrors human cheating when facing difficult or impossible tasks. This internal state, which increases as failur…