研究人员开发了MMDiff，一个增强扩散Transformer以实现多模态生成的新框架。该系统利用分布在整个去噪过程中的感知信息，使用轻量级解码器头联合生成图像和其他密集的感知模态。MMDiff在语义分割等任务上取得了显著改进，mIoU提高了28.7%，并展现出与DINOv3等最先进编码器相媲美的性能。 AI
影响 增强了扩散模型的多模态生成能力，可能改进合成数据生成和感知任务。
排序理由 该集群描述了一篇详细介绍生成模型新框架的研究论文。
AI 生成摘要 · Google Gemini · 来自 3 个来源。 我们如何撰写摘要 →
MMDiff transforms frozen diffusion transformers into multi-modal generative systems that produce images and perceptual modalities using lightweight decoders, achieving improved semantic segmentation through multi-timestep feature fusion and spatial aggregation.
arXiv:2606.16673v1 Announce Type: new Abstract: Diffusion transformers have demonstrated remarkable generative capabilities, yet the rich perceptual representations computed across their denoising trajectory are discarded once the content is rendered. We present MMDiff, a framewo…
Diffusion transformers have demonstrated remarkable generative capabilities, yet the rich perceptual representations computed across their denoising trajectory are discarded once the content is rendered. We present MMDiff, a framework that transforms a frozen diffusion transforme…