RISED: A Pre-Deployment Evaluation Framework for High-Stakes AI Decision-Support Systems, with Application to Healthcare
Researchers have developed RISED, a new framework for evaluating AI decision-support systems before deployment, particularly in high-stakes fields like healthcare. Unlike traditional methods that rely on a single accuracy metric, RISED assesses five critical dimensions: Reliability, Inclusivity, Sensitivity, Equity, and Deployability. When applied to various datasets spanning medical, credit, and income prediction, RISED revealed significant failures in inclusivity and sensitivity that were previously masked by aggregate accuracy scores, indicating that these issues are data-driven rather than model-specific. AI
IMPACT This framework could improve the safety and reliability of AI systems deployed in critical sectors like healthcare.