True Calibration Error

New method audits LLM calibration using logit bias

Researchers have developed a new method to audit the calibration of large language models (LLMs) when their continuous output probabilities are hidden. By manipulating the logit_bias parameter, a single query per sample…