Lightweight LLMs outperform rule-based systems in medical report labeling

A new study published on arXiv evaluated five lightweight, open-weight large language models (LLMs) for their ability to label chest, abdomen, and pelvis CT reports without prior fine-tuning. The LLMs, including MedGemma 27B and Gemma-3 27B, demonstrated superior performance compared to a rule-based algorithm and a fine-tuned RadBERT model. The research also highlighted that differences in labeling conventions between models and human annotators significantly impacted performance metrics. AI

IMPACT Lightweight, open-weight LLMs show promise for zero-shot medical report analysis, potentially reducing the need for extensive fine-tuning.

RANK_REASON The cluster contains an academic paper detailing research findings on LLM performance in a specific domain. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• Cohen kappa
• CT
• CT-RATE
• MedGemma-27B
• Michael Garcia-Alcoser

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →