OpenAI has demonstrated a GPT-3 model capable of expressing its uncertainty in natural language, without relying on internal model probabilities. The model can generate both an answer and a confidence level, such as "90% confidence," which are well-calibrated and maintain moderate calibration even when faced with shifts in data distribution. This research marks the first instance of a model verbally communicating calibrated uncertainty about its own responses and introduces a new testing suite called CalibratedMath. AI
RANK_REASON Academic paper detailing a new capability for language models.
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
