Researchers have developed JOVE, a novel framework designed to optimize the execution and verification of tasks distributed across various Large Language Models (LLMs). JOVE addresses the challenge of unknown LLM suitability for specific subtasks by jointly assigning executors and selecting intermediate outputs for verification. The system balances current execution costs against learning for future improvements under budget and latency constraints. By employing online learning to update LLM quality estimates based on verification feedback and incorporating an information-gain bonus, JOVE aims to improve allocation decisions. Experiments on four reasoning benchmarks show JOVE achieving competitive accuracy while reducing average cost and latency by at least 3.17 times compared to standard inference baselines. AI
IMPACT This framework could lead to more efficient and cost-effective use of LLMs for complex reasoning tasks, potentially lowering operational costs and improving response times.
RANK_REASON The cluster contains a research paper detailing a new framework for LLM task execution and verification. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
