per-token gates

New dual-path architecture scales LLM compute and capacity

Researchers have introduced a novel dual-path architecture for large language models designed to efficiently scale both compute and capacity. This architecture utilizes a deep sublayer applied multiple times with shared…