Malik et al.

Reward models show emergent misalignment after narrow harmful fine-tuning

Researchers have identified a phenomenon called "Emergent Misalignment" in reward models, where narrow fine-tuning for harmful preferences in one domain unexpectedly generalizes to other domains. This misalignment was o…