OLMo2

New theory models LLM training as noisy channel communication

Researchers have introduced the Shannon Scaling Law, a new theoretical framework for understanding Large Language Model (LLM) training. This model views LLM training as information transmission through a noisy channel, …