Trade-offs in Medical LLM Adaptation: An Empirical Study in French QA
A new study published on arXiv explores the effectiveness of different methods for adapting large language models (LLMs) to specialized domains and languages, using French medical question-answering as a case study. The research compares continual pretraining (CPT) and supervised fine-tuning (SFT), both individually and in combination, across various model families and sizes. Findings indicate that for multiple-choice questions, CPT+SFT often yields the best results, though SFT alone is a cost-effective alternative. For open-ended questions, CPT improves overlap-based metrics, while SFT can degrade quality, with instruction tuning and CPT+SFT being preferred by LLM-based evaluations. The study also demonstrates effective cross-lingual transfer from French to English benchmarks, offering practical guidance for adaptation strategy selection under computational constraints. AI
IMPACT Provides practical guidelines for optimizing LLM adaptation strategies, potentially reducing computational costs and improving performance in specialized domains.