Fiora Starlight

New RLHF method rewards AI red-teaming during training

A new research paper proposes Reinforcement Learning from Human Feedback (RLHF) that rewards red-teaming efforts within the training environment itself. This approach aims to improve AI safety by incentivizing the model…