A new study evaluated 15 advanced AI chatbots on their ability to perform emergency psychiatric triage using 112 clinical vignettes. The chatbots demonstrated high accuracy in identifying true emergencies, with an under-triage rate of only 5.6% for critical cases. However, they exhibited a tendency to over-triage less urgent situations, resulting in a mean signed ordinal error indicating a net over-estimation of urgency. Overall accuracy across all triage levels varied significantly between models, ranging from 42.0% to 71.8%. AI
Summary written by gemini-2.5-flash-lite from 1 source. How we write summaries →
IMPACT Highlights the need for improved calibration in AI models used for sensitive applications like mental health triage.
RANK_REASON Academic paper evaluating AI chatbot performance on a specific task.