Beyond Routing: Characterising Expert Tuning and Representation in Vision Mixture-of-Experts
Researchers have developed new methods to analyze the internal workings of Mixture-of-Experts (MoE) models in computer vision. Their work moves beyond simply examining how data is routed to specific "experts" within the model, instead focusing on what each expert actually encodes. The study found that an animate-inanimate distinction is a primary factor in how experts are partitioned, and this specialization is stable across different model initializations. AI
IMPACT Provides deeper insights into the internal representations of vision MoE models, potentially leading to more interpretable and robust AI systems.