A new research paper published on arXiv explores the limitations of current survival benchmark methodologies, particularly their reliance on the concordance index (C-index) and underrepresentation of deep learning (DL) methods. The study conducted a large-scale benchmark comparing 10 different methods across 49 datasets, treating the model selection criterion (C-index, Integrated Brier Score (IBS), or Mean Absolute Error (MAE-PO)) as a variable. Findings indicate that optimizing for IBS over C-index significantly improves probabilistic accuracy and calibration for complex DL and tree-based models. The research also highlights that dataset characteristics fundamentally govern model performance, with DL models scaling well with sample size but struggling in high-dimensional feature spaces, and notes a vulnerability where heavy censoring can artificially inflate calibration metrics. AI
IMPACT Challenges current evaluation standards for survival analysis, advocating for more transparent reporting of methodology and potentially influencing future benchmark design for deep learning models.
RANK_REASON The cluster contains a research paper detailing a large-scale benchmark and analysis of survival analysis methods. [lever_c_demoted from research: ic=1 ai=1.0]
- arXiv
- Bayesian mixed-effects modeling
- deep learning
- Hugging Face
- Integrated Brier Score
- Mae Poen
- mean absolute error
- X-Calakoop
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →