New framework learns pain cues from RGB facial videos, even with missing thermal/depth data

Researchers have developed ReMiX-MAE, a self-supervised multimodal masked pretraining framework designed to learn facial representations from synchronized RGB, thermal, and depth videos. This framework is specifically engineered to be robust to missing modalities, allowing for deployment using only RGB data. To support this research, a new dataset called the Sympathetic Mediated Pain (SMP) dataset was collected, featuring paired pre- and post-treatment recordings. Evaluations demonstrated that ReMiX-MAE outperforms RGB-only baselines, particularly in data-limited clinical scenarios, and shows improved transferability across external datasets. AI

IMPACT Enables more robust and data-efficient pain assessment in clinical settings by leveraging readily available RGB facial video.

RANK_REASON The cluster describes a new research paper detailing a novel framework and dataset for a specific AI application. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CV →

• arXiv
• Hugging Face
• ReMiX-MAE
• RGB color model
• Sympathetic Mediated Pain (SMP) dataset
• Thermal

• paper
• model release
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →