New StanceBench benchmark evaluates audio LLMs on interpersonal stance detection

Researchers have introduced StanceBench, a new benchmark designed to evaluate the interpersonal stance detection capabilities of audio-capable large language models (LLMs). This benchmark utilizes the Seamless Interaction corpus and defines nine distinct stance dimensions, standardizing both single-speaker and interaction-based evaluations. The study also assesses the robustness, bias, and stance inference accuracy of LLMs acting as automated judges, finding that empathy and politeness are the easiest stances to detect, while honesty proves to be the most challenging. AI

IMPACT Establishes a new evaluation standard for audio LLMs, potentially driving improvements in conversational AI's understanding of social nuance.

RANK_REASON The cluster contains an academic paper introducing a new benchmark for evaluating AI models. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• Seamless Interaction corpus
• StanceBench

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →