Researchers have developed a novel algorithm for identifying the optimal large language model (LLM) from a group, considering that each LLM has a different cost to query. The algorithm uses "dueling feedback," where pairwise comparisons of model responses provide preference signals, and incorporates heterogeneous sampling costs. This new approach, called Track-and-Stop, is designed to achieve asymptotically optimal cost as error decreases, and has shown consistent improvements over existing cost-unaware and cost-aware methods in evaluations on synthetic and real-world data. AI
IMPACT This research could lead to more efficient and cost-effective selection of LLMs for various applications.
RANK_REASON The cluster contains an academic paper detailing a new algorithm for LLM selection. [lever_c_demoted from research: ic=1 ai=1.0]
- alphaXiv
- arXiv
- CatalyzeX
- Condorcet criterion
- DagsHub
- Hugging Face
- IArxiv
- large-language models
- multi-armed bandit
- Track-and-Stop
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
