LLM Doesn't Know What It Doesn't Know: Detecting Epistemic Blind Spots via Cross-Model Attribution Divergence on Clinical Tabular Data
A new research paper explores the limitations of large language models (LLMs) when applied to structured clinical data, focusing on their inability to recognize their own knowledge gaps. The study found that LLM confidence scores are unreliable, often not correlating with accuracy. Furthermore, LLMs perform worse when traditional models like XGBoost are highly confident, but match performance when XGBoost is moderately uncertain. The research also demonstrated that few-shot examples and feature evidence are independent interventions that significantly improve accuracy and reduce attribution disagreement. AI
IMPACT Highlights the need for improved epistemic self-awareness in LLMs for reliable deployment in critical domains like healthcare.