Beyond Safe Data: Pretraining-Stage Alignment with Regular Safety Reflection
Researchers have introduced a new method called Safety Reflection Pretraining, designed to enhance the safety alignment of large language models (LLMs) during the pretraining phase. This approach goes beyond simply filtering or rewriting unsafe data by incorporating regular safety reflections into the pretraining corpora. Experiments with 1.7B models on the FineWeb-Edu dataset demonstrated improved safety classification accuracy and reduced susceptibility to attacks. A synthetic environment, MedSafetyWorld, was also developed to further validate the method's effectiveness in preventing models from generalizing unsafe behaviors from safe data. AI
IMPACT This research could lead to more robustly aligned LLMs, reducing risks associated with emergent unsafe behaviors.