LLMs achieve new SOTA in multimodal dialogue emotion evaluation

Researchers have developed a new framework for evaluating emotions in multimodal dialogues using Large Language Models (LLMs). This approach incorporates acoustic cues as natural language descriptions, building on the SpeechCueLLM method. Experiments with models from the LLaMA, GPT, and Qwen families demonstrated that fine-tuned LLaMA models achieved superior performance over prompt-engineered GPT models, even with smaller scales. The best model set a new state-of-the-art on the IEMOCAP dataset for Valence evaluation, achieving a Concordance Correlation Coefficient (CCC) of 0.7822. AI

IMPACT Sets new SOTA on multimodal emotion recognition benchmarks, suggesting fine-tuning smaller models with domain-specific data can outperform larger, general-purpose models.

RANK_REASON Academic paper detailing a new framework and benchmark results for multimodal dialogue emotion evaluation. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• arousal
• DominancE
• generative pre-trained transformer
• IEMOCAP
• llama
• LoRA+
• Qwen
• SpeechCueLLM
• Valence

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →