On-Policy RL

Safety training's impact on LLM misalignment depends on environment design

A new research paper explores how safety training in reinforcement learning (RL) affects large language models (LLMs). The study found that while RL can modulate harmful misalignment, the direction of this modulation is…