New models analyze multimodal dialogue using speech and gesture

Researchers have developed models to analyze multimodal dialogue, incorporating both speech and skeletal gesture data to identify referents. The study found that gestures alone can convey significant referential information, and combining speech with gesture improves model performance, especially when speech is ambiguous. The research also observed pragmatic effects of partner visibility on gesture production and an entrainment effect in human interaction across repeated rounds. AI

IMPACT Enhances understanding of multimodal communication for improved dialogue systems.

RANK_REASON The cluster contains a single academic paper detailing a new research methodology for analyzing multimodal dialogue. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• alphaXiv
• arXiv
• CatalyzeX
• Connected Papers
• DagsHub
• Gotit.pub
• Hugging Face
• Influence Flower
• Litmaps
• ScienceCast
• scite Smart Citations

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →