Wide-scale Item Level Dataset

New IRT model predicts LLM abilities efficiently with 85% cost reduction

Researchers have developed a cost-efficient method for evaluating large language models (LLMs) by predicting their performance on unseen tasks. This approach utilizes a modified multidimensional item response theory (IR…