Dendrograms of Mixing Measures for Softmax-Gated Gaussian Mixture of Experts: Consistency Without Model Sweeps
Researchers have developed a new statistical framework for Gaussian mixture of experts (SGMoE) models that addresses challenges in parameter estimation and model selection. The framework introduces novel loss functions and establishes convergence rates for maximum likelihood estimators, linking them to polynomial equation systems. For model selection, a dendrogram-based approach is proposed, which consistently identifies the number of experts without requiring multi-size training and demonstrates robustness to model misspecification. AI
IMPACT Introduces a more robust and efficient method for selecting the number of experts in SGMoE models, potentially improving their interpretability and performance in complex datasets.