LLMs show promise in identifying discourse units for aphasia assessment

A new research paper explores the use of instruction-tuned large language models (LLMs) for classifying Correct Information Units (CIUs) in aphasic discourse. The study found that while zero-shot prompting was insufficient, few-shot prompting significantly improved performance for models like Llama 3.1:8b, qwen2.5:7b, and mistral:7b, achieving competitive results with human annotators. However, the LLMs showed high recall but lower precision, indicating a tendency to over-classify tokens as CIUs, and performance varied with aphasia severity. AI

IMPACT LLM prompting shows potential for automated CIU identification in aphasia assessment, offering a human-in-the-loop solution.

RANK_REASON Research paper published on arXiv detailing LLM performance on a specific linguistic task. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• aphasia
• Cat Rescue
• Cohen's kappa
• Correct Information Units
• Llama 3.1:8b
• mistral:7b
• Nicholas and Brookshire
• Phi-3 Mini
• qwen2.5:7b

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →