value-distance kernels

New SMMD training method enhances numerical accuracy in LLMs

Researchers have developed a new training objective called Smooth Maximum Mean Discrepancy (SMMD) to improve the numerical precision of large language models (LLMs). Standard cross-entropy training treats numerical toke…