Brief

A new research paper published on arXiv examines the effectiveness of selective prediction methods for risk control in AI systems. The study found that common practices like naive thresholding can lead to a false sense of security, with error rates significantly exceeding declared budgets in many trials. Certified methods like Clopper-Pearson and betting upper confidence bounds showed better performance, but still experienced overruns under grouped deployment due to broken exchangeability premises. AI

This article explores the evolution of Markov's Inequality into a broader set of concentration-of-measure tools. It details how a single substitution within the inequality can lead to more powerful bounds like Chebyshev, Chernoff, Hoeffding, and Bernstein. The core technique involves applying a carefully chosen function to the original inequality. AI