Contextual Multidimensional IRT Model

New LLM evaluation framework predicts performance on unseen questions

Researchers have developed a novel framework for evaluating large language models (LLMs) that utilizes a multidimensional item response theory model combined with question contexts. This approach aims to predict LLM per…