Mixture-of-Transformers
PulseAugur coverage of Mixture-of-Transformers — every cluster mentioning Mixture-of-Transformers across labs, papers, and developer communities, ranked by signal.
4 day(s) with sentiment data
-
New datasets and methods boost robot learning with human egocentric videos
Researchers have introduced two new datasets aimed at improving robot learning through human egocentric videos. HubEgo-100K offers over 10,000 hours of diverse, in-the-wild egocentric video footage of skilled work, anno…
-
WeLM-Image model integrates multimodal understanding for advanced image generation
Researchers have introduced WeLM-Image, a unified image generation model designed for integration into large-scale consumer applications like WeChat. This model combines multimodal understanding with text-to-image gener…
-
WorldFlow model jointly predicts video and 3D motion via large-scale pretraining
Researchers have introduced WorldFlow, a novel generative world model designed to jointly predict future video and 3D trajectories. This model utilizes a Mixture-of-Transformers backbone to process both modalities simul…
-
New models FloMo and WorldFlow predict 3D motion from video
Researchers have developed FloMo and WorldFlow, two distinct models focused on predicting 3D scene motion and action from video. FloMo, a 5 billion parameter model, fine-tunes video generation capabilities to predict 3D…
-
RynnUnify: A Unified Embodied Model for Planning and Robot Control
Researchers have introduced RynnUnify, a 16-billion parameter unified embodied model designed to integrate visual planning, physical reasoning, and robot control. This Mixture-of-Transformers architecture builds upon Co…
-
RealtimeWAM achieves 25x speedup in world action models
Researchers have introduced RealtimeWAM, a novel approach to World Action Models (WAMs) designed for significantly improved inference efficiency. This method employs one-step action generation and asynchronous inference…
-
DriveWLA model unifies world, language, and action for autonomous driving
Researchers have introduced DriveWLA, a novel autonomous driving model that unifies semantic understanding, future scene prediction, and trajectory generation. This model utilizes a Mixture-of-Transformers architecture …
-
BridgeWAM decouples video and action experts for efficient action prediction
Researchers have developed BridgeWAM, a novel approach to World Action Models (WAMs) that decouples video and action experts using latent bridge queries. This method improves efficiency and generalization by allowing ac…
-
PixelUMM: Encoder-Free Unified Image and Video Model Introduced
Researchers have introduced PixelUMM, a novel encoder-free model designed for unified image and video understanding and generation directly in pixel space. This model overcomes the challenge of integrating separate visu…
-
AWS and NVIDIA launch Physical AI model factory with Cosmos 3
AWS and NVIDIA have collaborated to create a Physical AI model factory using NVIDIA Cosmos 3 on SageMaker HyperPod. This system is designed to continuously generate synthetic data, train perception and policy models, an…
-
New framework uses generation to boost MLLM visual understanding
Researchers have introduced GAS, a novel training framework that leverages generation as auxiliary supervision to enhance visual understanding in multimodal large language models (MLLMs). This approach adapts Next Embed…
-
Flex-π model integrates 3D geometry and object semantics with RGB data
Researchers have developed Flex-$\pi$, a 6-billion parameter world-action model that integrates 3D geometry and object semantics alongside RGB data. This model leverages a pre-trained video-generation VAE to encode 3D p…
-
New GAS framework enhances visual understanding in MLLMs with zero inference overhead
Researchers have developed a new framework called GAS that uses generation as an auxiliary supervision method to enhance visual understanding in multimodal large language models (MLLMs). GAS employs Next Embedding Predi…
-
StatePlay model generates mechanically consistent game worlds by predicting internal states
Researchers have introduced StatePlay, a novel game world model designed to generate visually realistic and mechanically consistent game environments. Unlike previous models that focus solely on pixel-level realism, Sta…
-
NVIDIA releases Cosmos 3 Edge, a 4B parameter on-device AI model for robots
NVIDIA has released Cosmos 3 Edge, a 4-billion-parameter open-world model designed for on-device operation in robotics and vision AI. This model enables robots and AI agents to understand their environment, reason in re…
-
New models RxBrain and GTA-VLA advance embodied AI reasoning
Researchers have introduced RxBrain, a novel foundation model for embodied cognition that integrates language and visual reasoning for planning. Unlike existing models that focus on scene understanding or future state p…
-
GigaWorld-Policy-0.5 enhances robot control with faster inference
Researchers have developed GigaWorld-Policy-0.5, an enhanced World Action Model (WAM) designed for more efficient robot control. This model addresses the computational overhead of traditional WAMs by using future visual…
-
NVIDIA Cosmos 3 World Models: Colab-Friendly Miniature Tutorial Released
This tutorial demonstrates how to implement a miniature version of NVIDIA's Cosmos 3 world models using the cosmos-framework, specifically tailored for Google Colab's hardware limitations. It guides users through checki…
-
New AI models advance tactile understanding and visuo-tactile manipulation
Researchers have developed UniTac, a novel unified multimodal model designed for tactile understanding and generation across different sensors. This model captures the physical interaction between sensors and objects th…
-
Mural integrates frozen LLMs into image generation via Mixture-of-Transformers
Researchers have developed a new method called Mural that integrates frozen Large Language Models (LLMs) with diffusion-based image generators. This approach utilizes a Mixture-of-Transformers (MoT) architecture to tran…