Biomedical AI research faces reproducibility crisis from LLM retirement

A new study published on arXiv highlights a significant reproducibility risk in biomedical research due to the retirement of large language models (LLMs). Researchers found that 42% of papers using LLMs involved models that were already retired or scheduled for retirement within two years of publication. The median time from publication to model retirement was 538 days, indicating that many biomedical publications relying on these models are likely to become computationally irreproducible shortly after being published. The study emphasizes that model deprecation needs to be addressed as a critical reporting and preservation issue in the field. AI

IMPACT This research highlights a critical need for better model lifecycle management and preservation strategies to ensure the long-term validity of scientific findings in AI-driven fields.

RANK_REASON The cluster contains a research paper published on arXiv detailing findings about LLM reproducibility in biomedical research. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• arXiv
• Biomedical AI Publications
• CatalyzeX Code Finder for Papers
• Connected Papers
• CORE Recommender
• DagsHub
• Gotit.pub
• Hugging Face
• Influence Flower
• Large language models
• Litmaps
• PubMed
• ScienceCast
• scite Smart Citations

• paper
• safety

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →