DistMoE

DistMoE enables rehearsal-free distributed tuning for multimodal LLMs

Researchers have introduced DistMoE, a novel mixture-of-experts approach designed for distributed visual instruction tuning of Multimodal Large Language Models (MLLMs). This method augments standard feedforward networks…