FutureOmni: Evaluating Future Forecasting from Omni-Modal Context for Multimodal LLMs
Researchers have introduced FutureOmni, a new benchmark designed to evaluate the future forecasting capabilities of multimodal large language models (MLLMs). The benchmark focuses on audio-visual environments and requires models to perform cross-modal reasoning and leverage internal knowledge to predict future events. Current MLLMs struggle with this task, with the best-performing model, Gemini 3 Flash, achieving only 64.8% accuracy. To address this, the researchers developed an instruction-tuning dataset and an Omni-Modal Future Forecasting (OFF) training strategy, which improved future forecasting and generalization. AI
IMPACT This benchmark and training strategy could lead to more capable multimodal models that can better understand and predict future events from complex data.