LLMs simulate plausible patients but fail to represent real populations

A new study published on arXiv reveals that large language models, when tasked with simulating mental health patients, produce individually plausible cases but fail to represent realistic populations. Models like GPT-4o mini, Gemini 3 Flash, DeepSeek-V3, and GLM-4.7 were tested on their ability to generate patient profiles based on demographic cohorts. While individual patient simulations largely adhered to diagnostic criteria, the aggregated populations showed significant deviations, including inflated symptom scores, distorted demographic disparities, and unstable regeneration of patient data. The researchers termed this discrepancy the "coherence-fidelity dissociation." AI

IMPACT Highlights potential risks in using LLMs for mental health simulations due to population-level inaccuracies.

RANK_REASON The cluster contains a research paper detailing findings on LLM capabilities. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• DeepSeek-V3
• DSM-5
• Gemini 3 Flash
• GLM-4.7
• GPT-4o mini
• Patrick Keough
• US National Health and Nutrition Examination Survey

• paper
• safety

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →