New RM-Net model improves multimodal sentiment prediction with missing data

Researchers have developed RM-Net, a novel neural network designed to improve multimodal sentiment prediction by accounting for missing data across text, audio, and video streams. The model represents unobserved temporal positions with a learned token and outputs bounded intensity contributions for each modality, modulated by observation quality. RM-Net was trained using dual-view masking of contiguous segments and achieved a macro-F1 score of 0.655 on complete inputs and 0.634 with 40% mixed missingness on a held-out split. AI

IMPACT This research could lead to more robust sentiment analysis systems that can function effectively even with incomplete multimodal data.

RANK_REASON The cluster contains an academic paper detailing a new model and its performance on a specific task. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CV →

• arXiv
• Bert
• CMU-MOSEI
• LoRA+
• RM-Net

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →