Researchers have found that extending latent Chain-of-Thought (CoT) reasoning in large language models to a fixed horizon can sometimes decrease performance. The study identifies distinct stages in latent CoT, where early computations are crucial for reasoning, while later stages may reorganize existing information and weaken accuracy. This suggests a need to rethink fixed-horizon inference. The researchers propose a method to estimate remaining reasoning progress and adaptively adjust the latent horizon during inference, which has shown improvements of up to 7.2 percentage points in reasoning accuracy while reducing computation by up to 29.0% on arithmetic benchmarks, without requiring LLM retraining. AI
IMPACT This research could lead to more efficient and accurate LLM inference by dynamically adjusting reasoning horizons.
RANK_REASON This is a research paper detailing novel findings and methods related to LLM reasoning. [lever_c_demoted from research: ic=1 ai=1.0]
- alphaXiv
- arXiv
- CatalyzeX Code Finder for Papers
- Connected Papers
- CORE Recommender
- DagsHub
- Gotit.pub
- Hugging Face
- IArxiv Recommender
- Influence Flower
- Litmaps
- ScienceCast
- scite Smart Citations
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →