From Memorization to Parameter Interference: How Overtraining Experts Harms Model Merging
A new research paper from arXiv explores how overtraining expert models can negatively impact the effectiveness of model merging. The study, which examined vision and language modalities across various model scales and adaptation methods like LoRA, found that fine-tuning models too extensively on difficult examples leads to memorization. This memorization causes parameter interference, resulting in degraded performance when merging these overtrained experts. The researchers propose task-dependent early stopping as a strategy to mitigate this issue and improve merging outcomes. AI
IMPACT Overtraining expert models can degrade performance when merging them, suggesting a need for careful fine-tuning strategies and early stopping to maximize combined model capabilities.