Study: LLMs sound persuasive but argue poorly in dialogues

A new study published on arXiv introduces Persuasio, a platform designed to evaluate the persuasive dialogue capabilities of large language models (LLMs). The research generated 192 debates on a UK political topic, pitting humans against LLMs. Findings indicate a significant gap between LLMs' perceived persuasiveness and their actual argumentative strength, with humans remaining competitive in formal adjudication despite LLMs dominating subjective rankings. AI

IMPACT Highlights a potential disconnect between LLM fluency and logical reasoning, suggesting current models may not be as adept at formal argumentation as they appear.

RANK_REASON Research paper published on arXiv detailing a new evaluation method for LLMs. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• arXiv
• LLMs
• UK

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →