Fluid Benchmarking

LLM evaluation method adapts psychometric theory for fairer benchmark comparisons

Researchers have developed a new method for evaluating large language models (LLMs) by adapting Item Response Theory (IRT) from psychometrics. This approach treats each benchmark as an "item" and models the LLM's "abili…