Researchers have developed SheafPrune, a novel method for reducing the computational cost of multimodal large language models by pruning irrelevant visual tokens. This technique utilizes a graph-based approach inspired by sheaf geometry to propagate relevance scores between text and visual tokens, correcting for node degree to improve accuracy. SheafPrune, with approximately 4.5 million parameters, is trained end-to-end with the LLaVA-NeXT-8B backbone and can adapt to various token budgets. Experiments show that SheafPrune significantly speeds up prefill times and reduces FLOPs while maintaining high accuracy across multiple benchmarks. AI
IMPACT Reduces inference costs for multimodal LLMs, potentially enabling wider deployment and faster processing.
RANK_REASON Academic paper detailing a new method for optimizing LLM inference. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
