Brief

Researchers have developed a new routing mechanism for Mixture-of-Experts (MoE) models called kNN-MoE. This approach uses a memory of past routing decisions to dynamically assign tokens to experts, improving robustness against distribution shifts. The system leverages k-nearest neighbors to find similar past cases and uses their similarity as a confidence score for mixing expert assignments. Experiments indicate that kNN-MoE performs better than standard zero-shot methods and is comparable to supervised fine-tuning. AI

Researchers have developed a new method for detecting errors in machine translation that does not require human annotation. This approach, called Iterative MBR Distillation, uses a large language model to generate its own training data, effectively creating pseudo-labels. Experiments show that models trained with this self-generated data perform better than those trained on human-annotated datasets, particularly at identifying specific error spans. AI