New multimodal models and research push AI capabilities across edge, reasoning, and generation
ByPulseAugur Editorial·[27 sources]·
Several research papers and a new model release focus on advancing multimodal AI capabilities. Liquid Foundation Models has released the open-source d1-3B model, optimized for edge devices and capable of processing text and images with fast inference times. Concurrently, new research explores methods for improving multimodal learning under noisy supervision (REFINE), benchmarking intuitive visual reasoning (Humanity's Sixth Sense), structured specialization for omnimodal embeddings (Syn-Omni), learning multimodal synergy via tensor modeling (HRIL), and weaving visual world modeling into LLMs (WOVEN). Additionally, MUNITE offers a unified framework for any-to-any multimodal generation, and PrismSSL provides a library for multimodal self-supervised learning.
AI
IMPACT
Advances in multimodal AI are improving reasoning, generation, and efficiency, enabling more sophisticated applications across various domains.
RANK_REASON
Multiple research papers on multimodal AI advancements published on arXiv.
arXiv:2610.12417v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) struggle with spatial, embodied, physical, and temporal reasoning. We hypothesize that these failures reflect a shared deficit in visual transition reasoning, and test whether this capabili…
arXiv cs.AI
TIER_1English(EN)·Xingang Guo, Jing Gu, Brian Jang, Renxiong Wang, Utkarsh Tyagi, Daniel Quigley, Steven Li, David Yan, Daniel Yue Zhang, Darvin Yi, Forrest Huang, HiJae Kim, Tianyi Zhang, Jared Lichtarge, Jihua Huang, Le Xue, Manan Tomar, Qiuyi Richard Zhang, Ruofei Yu, …·
arXiv:2610.08966v2 Announce Type: replace Abstract: Humans perceive far more in a scene than what is explicitly depicted: a single glance captures past causes and future trajectories; a quick peek determines if a vehicle can fit between two parked cars; a few seconds of video rev…
arXiv:2610.11057v1 Announce Type: cross Abstract: Multimodal classification processes and relates information from multiple modalities to achieve more accurate predictions. However, existing methods typically rely on high-quality ground-truth labels, which are difficult to obtain…
arXiv:2610.12393v1 Announce Type: new Abstract: Self-supervised multimodal representation learning has achieved remarkable success across diverse domains, yet capturing synergistic information remains challenging due to the complexity of cross-modal interactions. Unlike the share…
arXiv:2610.12256v1 Announce Type: cross Abstract: Omnimodal embeddings naturally involve both shared representations and modality-specific features across heterogeneous inputs. However, existing omnimodal embedding methods often rely on a single shared parameter space over mixed-…
Omnimodal embeddings naturally involve both shared representations and modality-specific features across heterogeneous inputs. However, existing omnimodal embedding methods often rely on a single shared parameter space over mixed-modality data, limiting structural separation betw…
arXiv:2610.09866v1 Announce Type: new Abstract: We introduce MUNITE, a latent-variable framework for flexible any-to-any multimodal generation that treats encoding and latent generation as the same inference problem under different amounts of observed evidence. Given any subset o…
arXiv:2511.17776v2 Announce Type: replace Abstract: We present PrismSSL, a Python library that unifies state-of-the-art self-supervised learning (SSL) methods across audio, vision, graphs, and cross-modal settings in a single, modular codebase. The goal of the demo is to show how…
Single-image, multi-image, and video deep research require different visual operations but share a workflow of visual grounding, external retrieval, and fact composition. A key challenge is to preserve the dependencies linking localized visual anchors, entity relations, source-su…
arXiv cs.LG
TIER_1English(EN)·Payal Mohapatra, Haodong Yang, Yueyuan Sui, Stephen Xia, Benjamin Lundell, Qi Zhu·
arXiv:2610.07466v1 Announce Type: new Abstract: Multimodal systems often encode every available input, even when a subset suffices for prediction. Adaptive acquisition can reduce this cost by using predictions from incrementally fused evidence to decide which modality to encode n…
arXiv:2610.07887v1 Announce Type: cross Abstract: Unified multimodal models (UMMs) integrate understanding and generation, yet their generative behavior is rarely governed by what they understand about the task. We formalize visual abstention: when a requested visual transformati…
arXiv:2610.07928v1 Announce Type: cross Abstract: Dynamic multimodal learning aims to learn robust representations by adaptively modeling information discrepancies across modalities. However, existing methods still suffer from two limitations: (i) static cross-modal alignment str…
arXiv:2602.02140v2 Announce Type: replace Abstract: Recent advances in unified multimodal models (UMM) have demonstrated remarkable progress in both understanding and generation tasks. However, whether these two capabilities are genuinely aligned and integrated within a single mo…
Multimodal large language models have become the dominant paradigm for visual understanding, but incur substantial costs by encoding inputs into dense, fixed-size patch tokens. However, visual information is unevenly distributed: some regions require fine-grained detail, while ot…
arXiv:2610.03002v1 Announce Type: new Abstract: Unified multimodal models (UMMs) understand and generate both text and images, which lets a model produce its own training data. Existing self-improvement in UMMs keeps supervision on the visual side, where image understanding judge…
arXiv:2610.03035v1 Announce Type: new Abstract: Multimodal learning often suffers from modality imbalance, where the joint optimization process is dominated by a single modality. Existing methods typically estimate modality imbalance from score disparities derived from prediction…
Few-step streaming audio--video generation requires both causal modeling and step distillation, yet standard training recipes face two context-related challenges. Teacher forcing pairs clean history with a noisy target, but supervises predictive contextual representations only in…
Multimodal large language models can expose task-relevant evidence through generation, but producing useful evidence does not by itself determine how it enters a retrieval embedding. We study whether the semantic organization of that evidence can also specify where representation…
arXiv:2610.12060v1 Announce Type: new Abstract: Processing long visual token sequences from high-resolution images makes multi-step reasoning computationally expensive for multimodal Large Language Models (MLLMs). Existing one-shot pruning and aggregation methods compress visual …
arXiv cs.CV
TIER_1English(EN)·Hongyu Li, Manyuan Zhang, Kaituo Feng, Shu Chen, Dian Zheng, Hao Li, Hao Yu, Zhangquan Chen, Zoey Guo, Ray Zhang, Shaofei Huang, Tianrui Hui, Linjiang Huang, Si Liu·
arXiv:2610.12419v1 Announce Type: new Abstract: Single-image, multi-image, and video deep research require different visual operations but share a workflow of visual grounding, external retrieval, and fact composition. A key challenge is to preserve the dependencies linking local…
arXiv:2603.06578v3 Announce Type: replace Abstract: Multimodal Large Language Model (MLLM) classification performance depends critically on evaluation protocol and ground truth quality. Studies comparing MLLMs with supervised and Vision-Language Models (VLMs) report conflicting c…
arXiv cs.CV
TIER_1English(EN)·Naim Reza, Md Al Amin, Ho Yub Jung·
arXiv:2610.11117v1 Announce Type: new Abstract: Dynamic convolution enhances convolutional neural networks (CNNs) by adapting kernels to input content, but it expresses the effective kernel as a linear mixture of a small number of basis kernels, which limits expressivity and comp…
arXiv:2610.10133v1 Announce Type: new Abstract: Real-world low-quality images suffer from complex mixed degradations, including but not limited to noise, blur, atmospheric effects, etc. Recent agentic methods usually model real-world image restoration (Real-IR) as a sequential to…
<p>Cloudflare released Clef-omni, an Apache-2.0 multimodal decision model built on Qwen3-Omni-30B-A3B. It scores text, images, audio and video in one call for $0.15 per million input tokens, while Clef-flash drops to $0.038.</p> <p>The post <a href="https://www.marktechpost.com/2…
<p>Dealing with complex tasks that require understanding both text and images usually means hitting external APIs, racking up costs, and dealing with potential data privacy concerns. Building intelligent agents that can interpret visual information alongside natural language, and…