manifold hypothesis

Tropical geometry reveals sparsity is combinatorial depth in MoE models

A new paper introduces a theoretical framework for understanding Mixture-of-Experts (MoE) models using tropical geometry. The research establishes that the routing mechanism in MoE architectures is equivalent to a speci…