NormBT

New NormBT method improves LLM reward model training

Researchers have identified a bias in the Bradley-Terry (BT) loss function commonly used for training reward models in LLM alignment. This bias stems from representation distance, where pairs of responses with large dis…