Perplexity is a metric used to evaluate language models by measuring how surprised the model is by a given piece of text. A lower perplexity score indicates that the model found the text more predictable and thus better understands it, akin to rolling a die with fewer sides. The metric is calculated by averaging the negative logarithm of the probability assigned to the true next word at each position in the text and then exponentiating the result. AI
IMPACT Provides a foundational understanding of a key metric for evaluating language model performance.
RANK_REASON The item explains a core concept in language model evaluation (perplexity) and provides a tutorial on how to measure it, including code examples. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
