New AI model CALM-AH improves recognition of ambivalence and hesitancy in videos

Researchers have developed CALM-AH, a multimodal ensemble system designed to recognize ambivalence and hesitancy in video interviews. The system integrates textual, acoustic, visual, and behavioral-statistical features, employing a novel Reliability-Gated Multi-Expert Consensus (RG-MEC) approach. This method combines an initial prediction with three correction experts, only overriding the default prediction if all correction experts unanimously agree on an alternative class, thereby limiting isolated errors while allowing for consistent bidirectional correction. CALM-AH achieved a Macro-F1 score of 0.7525, and the full RG-MEC system reached 0.7771 on the ABAW11 dataset. AI

IMPACT This research introduces a novel ensemble method for recognizing subtle human emotional states in video, potentially improving human-AI interaction and analysis tools.

RANK_REASON This is a research paper detailing a new model and method for a specific recognition task. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CV →

• ABAW11
• AffectGPT
• brother
• generative pre-trained transformer
• RG-MEC

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →