Researchers have identified a paradox with Power Sampling, a technique used to improve language model reasoning. While it can concentrate probability mass towards correct answers, it paradoxically leads to worse overall inference performance, with accuracy drops of up to 18.5 percentage points observed on reasoning benchmarks. This issue stems from "dose mismatch," where a fixed exponent causes varying distributional changes across problems, and "coverage mismatch," where global sharpening narrows the focus to dominant paths, potentially losing broad reasoning support. A proposed solution involves a deformation-controlled, support-preserving Power target that calibrates sharpening and limits the suppression of moderate-probability paths, outperforming standard multi-sample inference. AI
IMPACT This research could lead to more reliable and accurate reasoning in large language models by addressing a counterintuitive flaw in current sampling methods.
RANK_REASON Research paper detailing a novel finding and proposed solution for a language model inference technique. [lever_c_demoted from research: ic=1 ai=1.0]
- alphaXiv
- arXiv
- CatalyzeX
- DagsHub
- Gotit.pub
- Hugging Face
- IArxiv
- Influence Flower
- Power Sampling
- ScienceCast
- Self Consistency In Llms
- weighted self-consistency
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
