PulseAugur
中
实时 15:49:13

New DMEP framework prunes LoRA-MoE experts for better efficiency and accuracy

作者 PulseAugur 编辑部 · [2 个来源] ·

Researchers have developed a new framework called DMEP for efficient fine-tuning of LoRA-MoE models. This method dynamically prunes low-utility experts on a per-module basis, creating a more compact and specialized model structure. By removing the load-balancing constraint after initial training, DMEP allows remaining experts to specialize further. Experiments show DMEP reduces trainable parameters by up to 43% and increases training throughput by about 10% while maintaining accuracy. AI

影响 Reduces trainable parameters and improves training efficiency for LoRA-MoE models, potentially lowering fine-tuning costs.

排序理由 This is a research paper detailing a new method for efficient fine-tuning of existing model architectures.

在 arXiv cs.LG 阅读 →

AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →

New DMEP framework prunes LoRA-MoE experts for better efficiency and accuracy

报道来源 [2]

  1. arXiv cs.LG TIER_1 English(EN) · Weihang Li, Jianchun Liu, Hongli Xu ·

    Adaptive and Fine-grained Module-wise Expert Pruning for Efficient LoRA-MoE Fine-Tuning

    arXiv:2604.26340v1 Announce Type: new Abstract: LoRA-MoE has emerged as an effective paradigm for parameter-efficient fine-tuning, combining the low training cost of LoRA with the increased adaptation capacity of Mixture-of-Experts (MoE). However, existing LoRA-MoE frameworks typ…

  2. arXiv cs.LG TIER_1 English(EN) · Hongli Xu ·

    Adaptive and Fine-grained Module-wise Expert Pruning for Efficient LoRA-MoE Fine-Tuning

    LoRA-MoE has emerged as an effective paradigm for parameter-efficient fine-tuning, combining the low training cost of LoRA with the increased adaptation capacity of Mixture-of-Experts (MoE). However, existing LoRA-MoE frameworks typically adopt a fixed and uniform expert configur…

相关实体

相关话题