Multi-Rate Mixture of Experts for Accelerating Liquid Neural Network Training
Researchers have introduced a Multi-Rate Mixture-of-Experts (MR-MoE) framework designed to enhance Liquid Neural Networks (LNNs). This new architecture allows multiple LNN experts to operate at different time scales, enabling the model to better distinguish between rapid and slow temporal patterns in multivariate time-series data. The framework also incorporates gating networks for adaptive expert selection and attention mechanisms for improved robustness and long-range dependency modeling. Experiments show that the MR-MoE approach outperforms traditional LSTMs and standard MoE models in time-series prediction tasks, achieving better AUROC and AUPRC scores with comparable computational efficiency. AI
IMPACT Introduces a novel architecture for time-series modeling that could improve performance and efficiency in complex sequential data tasks.