Bayes-optimal next-token predictor

New theory explains cross-lingual similarity in language model representations

Researchers have developed a theoretical framework to explain the phenomenon of similar representations for translated sentences in the inner layers of multilingual language models. This framework is based on the idea t…