New metric quantifies OOD score instability in AI models

A new paper titled "Verdict Instability of OOD Scores under Reference Resampling" introduces a metric called verdict instability to quantify the variability of out-of-distribution (OOD) scores when the reference dataset is resampled. This instability is calculated as the within-class dispersion of the assigned class along the query's direction, divided by the square root of that class's reference count. The research indicates that estimators of local dispersion carry the expected sign for practitioners, and abstention based on incorrectly signed scores can be worse than random abstention. AI

IMPACT Introduces a new metric for evaluating the reliability of out-of-distribution detection in AI models.

RANK_REASON The cluster contains a research paper detailing a new metric for evaluating AI model performance. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv stat.ML →

• alphaXiv
• arXiv
• arXivLabs
• CatalyzeX Code Finder for Papers
• CORE Recommender
• DagsHub
• Gotit.pub
• Hugging Face
• IArxiv Recommender
• Influence Flower
• O'odham
• ScienceCast
• Verdict Instability of OOD Scores under Reference Resampling

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →