Honglu Fan

Transformer embedding dimension strengthens internal world models in sorting tasks

A new research paper explores how the embedding dimension in transformer models impacts the development of internal world models when trained on a simple sorting task. The study found that while models can achieve high …