Self-consistency technique shows diminishing returns for modern LLMs

A new study suggests that the self-consistency technique, which involves generating multiple reasoning paths to improve LLM accuracy, is becoming less effective and more costly. Researchers found minimal accuracy gains on benchmarks like HotpotQA and MATH-500 when increasing the number of samples, while token costs rose linearly. In some cases, performance even declined with more samples, indicating that self-consistency may introduce noise rather than signal for modern, more capable LLMs. AI

IMPACT Suggests that traditional self-consistency methods may be inefficient for advanced LLMs, potentially impacting inference cost optimization strategies.

RANK_REASON Academic paper analyzing the diminishing returns of a specific LLM technique. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• Chiyan Loo
• HotpotQA
• LLMs
• MATH-500
• Gemini 2.5

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →