研究人员推出了两个新框架，以改进大语言模型的多模态指令调优。SAME框架通过稳定专家选择和规范专家更新来解决持续学习中的“路由器漂移”和“专家漂移”问题。同时，OFA框架提供了一种可重用的数据选择方法，只需训练一次选择器即可将其应用于各种数据集和模型，通过仅选择一小部分数据即可显著提高训练效率，同时保持高性能。此外，Prism基础设施提供了一个插件系统，以简化多模态持续指令调优的研发，将算法开发与基础模型实现分离，以增强代码重用和公平比较。 AI
影响 这些进展有望实现更高效、可扩展的多模态大语言模型训练，使其能够持续适应新任务和数据。
排序理由 多篇研究论文介绍了用于多模态持续指令调优的新框架和基础设施。
AI 生成摘要 · Google Gemini · 来自 6 个来源。 我们如何撰写摘要 →
arXiv:2602.01990v2 Announce Type: replace-cross Abstract: Multimodal Large Language Models (MLLMs) achieve strong performance through instruction tuning, but real-world deployment requires them to continually expand their capabilities, making Multimodal Continual Instruction Tuni…
Multimodal instruction tuning is the de facto recipe for adapting vision language models (VLMs), yet instruction data are highly redundant, making data selection critical for training efficiency. Existing methods derive selection signals from a specific model or dataset, so whene…
arXiv:2605.26110v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) achieve versatility by reformulating diverse tasks into a unified instruction-following framework via instruction tuning. However, real-world deployment requires continuous adaptation to em…
Multimodal Large Language Models (MLLMs) achieve versatility by reformulating diverse tasks into a unified instruction-following framework via instruction tuning. However, real-world deployment requires continuous adaptation to emerging tasks, motivating Multimodal Continual Inst…
arXiv:2605.26761v1 Announce Type: new Abstract: Multimodal instruction tuning is the de facto recipe for adapting vision language models (VLMs), yet instruction data are highly redundant, making data selection critical for training efficiency. Existing methods derive selection si…
Multimodal instruction tuning is the de facto recipe for adapting vision language models (VLMs), yet instruction data are highly redundant, making data selection critical for training efficiency. Existing methods derive selection signals from a specific model or dataset, so whene…