ARBITER method improves LLM accuracy by analyzing reasoning trajectory basins

Researchers have introduced ARBITER, a novel method designed to improve the accuracy of language models by addressing failures in test-time sampling. Standard sampling methods generate multiple reasoning paths and select an answer via majority vote, but this can lead to incorrect answers being chosen if they belong to the most stable, rather than most accurate, reasoning basin. ARBITER models interactions between these basins using only the model's own sampled outputs and hidden states, enhancing consensus with additive evidence. This approach has demonstrated consistent accuracy gains across various models and benchmarks, recovering a significant portion of potential performance improvements without external information. AI

IMPACT Enhances LLM reasoning by mitigating majority vote failures in sampling, potentially improving accuracy on complex tasks.

RANK_REASON This is a research paper detailing a new method for improving language model performance. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• GSM8K
• Llama-3.1-8B
• MMLU-HS-Math
• Qwen3-4B

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →