New text-centric post-training boosts multi-modal LLM reasoning

Researchers have developed a text-centric post-training method to improve multi-modal reasoning in large language models, specifically focusing on models that process text and audio-visual data. This approach involves initial text-only reasoning training, which significantly boosts reasoning scores while reducing computational costs. A subsequent refinement stage uses a reduced dataset of audio-visual data to enhance perception without substantial loss of reasoning gains. This method has shown to improve the Qwen2.5-Omni-7B model's reasoning capabilities by over 25% compared to its base version, using fewer GPU hours. AI

IMPACT This approach could lead to more efficient training of multi-modal AI systems, reducing computational costs and improving their reasoning abilities.

RANK_REASON The cluster contains an academic paper detailing a new method for improving LLM capabilities. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• arXiv
• Hugging Face
• Omni Large Language Models
• Qwen2.5-Omni-7B

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →