Chiwun Yang

Transformer scaling law theory unifies learning dynamics and generalization

Researchers have developed a theoretical framework to unify the understanding of learning dynamics and generalization in transformer models. This work formalizes transformer training as an ordinary differential equation…