研究人员推出了TorchUMM，一个用于评估、分析和后训练各种统一多模态模型（UMMs）的统一代码库。该框架旨在通过提供通用接口和评估协议，来标准化不同UMM架构和任务（包括理解、生成和编辑）之间的比较。另外，Lance模型通过多任务协同提供了一种轻量级的统一多模态建模方法，侧重于协同训练而非单纯的模型容量。Lance利用双流专家混合架构和分阶段多任务训练，以增强图像和视频的理解和生成能力。 AI
影响 标准化的评估框架和新颖的建模方法有望加速统一多模态AI系统的进步。
排序理由 两篇研究论文介绍了用于统一多模态AI的新代码库和模型。
在 Hugging Face Daily Papers 阅读 →
AI 生成摘要 · Google Gemini · 来自 3 个来源。 我们如何撰写摘要 →
arXiv:2604.10784v2 Announce Type: replace Abstract: Recent advances in unified multimodal models (UMMs) have led to a proliferation of architectures capable of understanding, generating, and editing across visual and textual modalities. However, developing a unified framework for…
arXiv:2605.18678v2 Announce Type: replace-cross Abstract: We present Lance, a lightweight native unified model supporting multimodal understanding, generation, and editing for both images and videos. Rather than relying on model capacity scaling or text-image-dominant designs, La…
LatentUMM addresses multimodal consistency issues by constructing an enhanced shared latent space that explicitly aligns transformations between modalities and stabilizes latent dynamics during generation and re-encoding processes.