annotation-anchored training

New training method combats LLM diversity loss

Researchers have developed a new method called annotation-anchored training to address semantic mode collapse in large language models. This technique involves pretraining models on documents paired with semantic annota…