New MIRAGE framework enhances fMRI encoding with multimodal gating

Researchers have developed MIRAGE, a new framework for encoding whole-brain fMRI responses to naturalistic audiovisual stimuli. This model utilizes a native multimodal backbone and adaptive feature gating across layers to integrate visual, auditory, and linguistic information. MIRAGE outperforms existing methods by natively combining multimodal features rather than aggregating unimodal ones post-hoc, offering improved predictive accuracy and interpretable modality-specific gating patterns across the cortex. AI

IMPACT Introduces a novel approach to integrating multimodal data for brain response prediction, potentially advancing neuroscience research and AI's understanding of sensory processing.

RANK_REASON This is a research paper detailing a new framework for fMRI encoding. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• fMRI
• MIRAGE

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →