LLMs struggle with multi-turn evidence gathering for reasoning, study finds

A new paper titled "Don't Let Me Ask for It: LLMs Show Deficiencies in Active Multi-Turn Information Acquisition for Abductive Inference" introduces the "Alien Abduction game" to test large language models' (LLMs) ability to acquire evidence and refine hypotheses in multi-turn abductive reasoning scenarios. The study found that LLMs perform better when all evidence is provided upfront rather than across multiple turns. Additionally, models that receive oracle-provided examples achieve higher success rates than those that select their own queries, though their final hypotheses are more consistent with self-selected evidence. AI

IMPACT This research highlights limitations in LLMs' ability to actively seek and integrate information over time, suggesting potential issues for complex reasoning tasks.

RANK_REASON The cluster contains an academic paper detailing research findings on LLM capabilities. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• Alien Abduction game
• Kranti Chalamalasetti
• LLMs

• paper
• safety

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →