New benchmark reveals mixed clinical utility for EHR foundation models

A new benchmark called FoMoH has been developed to evaluate foundation models (FMs) for structured electronic health records (EHRs). This benchmark includes 14 clinically meaningful prediction tasks and was tested on over 6 million patient records from Columbia University Irving Medical Center and MIMIC-IV. The evaluation found that while top-performing FMs outperform traditional models in discriminative performance, especially with limited labeled data, they may underperform in low-prevalence settings and exhibit lower calibration. Cross-institutional transportability also remains a challenge for these EHR FMs. AI

IMPACT Highlights limitations in current EHR foundation models, guiding future research towards improved calibration and cross-institutional applicability.

RANK_REASON The cluster contains an academic paper introducing a new benchmark for evaluating foundation models in a specific domain (EHRs). [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• Columbia University Irving Medical Center
• FoMoH
• MIMIC-IV
• Vincent Jeanselme

• paper
• model release
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →