New HART technique enables LMMs to reason with high-resolution images without annotations

Researchers have developed a new technique called HART (High-resolution Annotation-free Reasoning Technique) to improve how Large Multimodal Models (LMMs) handle high-resolution images. Current LMMs struggle with the large number of tokens generated by high-resolution images, often requiring costly human annotations to identify important regions. HART uses a closed-loop framework and a policy optimization method called AP-GRPO to enable LMMs to self-verify key regions without external supervision. Experiments across several benchmarks show that HART significantly enhances performance on high-resolution visual tasks. AI

IMPACT This technique could reduce the cost and complexity of training AI models for high-resolution image analysis, potentially leading to more capable visual AI systems.

RANK_REASON The cluster describes a new research paper detailing a novel technique for improving AI model performance on specific tasks. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CV →

• AP-GRPO
• HART
• HR-Bench-4K/8K
• Jiacheng Yang
• Large Multimodal Models
• MME-RealWorld-Lite
• MMStar
• TreeBench
• V* Bench

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →