New MWOP technique boosts MLLM efficiency with targeted pruning

Researchers have developed a new method called Modality-aware Width-wise Operation Pruning (MWOP) to improve the efficiency of multimodal large language models (MLLMs). MWOP addresses redundancy within attention heads and feed-forward network (FFN) channels by independently pruning visual-to-visual, text-to-visual, and text-to-text attention paths, and separately selecting FFN channels for visual and textual inputs. This approach preserves token sequences while reducing computation, leading to significant speedups. On LLaVA-OneVision-7B, MWOP alone achieved a 1.6x prefill speedup with minimal performance loss, and when combined with token compression methods, further increased speedups. AI

IMPACT This method could lead to more efficient deployment and faster inference for multimodal AI systems.

RANK_REASON Research paper detailing a new method for model efficiency. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• arXiv
• Hugging Face
• LLaVA-OneVision-7B
• LoRA+
• MLLMs
• Qwen2.5-VL-7B
• Triton

• paper
• infra

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →