Researchers have developed ET-Prune, a novel framework designed to optimize the inference costs of multimodal large language models (MLLMs) by dynamically pruning visual tokens. Unlike fixed pruning ratios, ET-Prune adapts to text-rich inputs by casting pruning as evidence allocation, identifying and safeguarding crucial information like small numbers or labels specified by a query. This method converts evidence uncertainty and density into a sample-specific token budget, leading to significant improvements in quality-cost trade-offs. In evaluations on OCRBench-v2 and MMBench v1.1, ET-Prune achieved competitive or leading results among pruned methods while retaining approximately half of the visual tokens. AI
IMPACT This method could significantly reduce the computational cost of running multimodal AI models, making them more accessible and efficient for various applications.
RANK_REASON The cluster contains a research paper detailing a new method for multimodal large language models. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
