Researchers have developed a new method for selecting subsets of prompts for Large Language Model (LLM) benchmarks, aiming to approximate the results of full benchmark suites with significantly fewer prompts. This evaluation-unsupervised approach utilizes submodular subset selection, with facility location functions operating on semantic prompt embeddings proving most effective. The method was tested on a large dataset of 35 benchmarks, 18 LLMs, and over 61,000 prompts, demonstrating superior performance compared to existing baselines in preserving LLM scores. AI
IMPACT This research could lead to more efficient and cost-effective LLM evaluation by reducing the number of prompts needed for benchmarking.
RANK_REASON The item is an academic paper detailing a new methodology for LLM benchmark evaluation. [lever_c_demoted from research: ic=1 ai=1.0]
- alphaXiv
- arXiv
- CatalyzeX
- DagsHub
- Gotit.pub
- Hugging Face
- Massive Multitask Language Understanding
- MTEB
- ScienceCast
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
