Self-consistency technique shows diminishing returns for modern LLMs

A new study suggests that the self-consistency technique, which involves generating multiple reasoning paths to improve LLM accuracy, is becoming less effective and more costly. Researchers found minimal accuracy gains on benchmarks like HotpotQA and MATH-500 when increasing the number of samples, while token costs rose linearly. In some cases, performance even declined with more samples, indicating that self-consistency may introduce noise rather than signal for modern, more capable LLMs. AI

Summary written by gemini-2.5-flash-lite from 1 source. How we write summaries →

IMPACT Suggests that traditional self-consistency methods may be inefficient for advanced LLMs, potentially impacting inference cost optimization strategies.

RANK_REASON Academic paper analyzing the diminishing returns of a specific LLM technique. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• Self-Consistency
• LLMs
• Gemini 2.5
• HotpotQA
• MATH-500
• Chiyan Loo

• paper
• other