New LLM method uses hidden-state geometry for better reasoning

Researchers have developed Cloud-ScPO, a novel framework for semi-supervised preference optimization in large language models (LLMs) that leverages the geometric structure of internal model states. This method uses a small labeled dataset to construct reference 'Clouds' of correct and incorrect reasoning trajectories. By analyzing the connectivity and spatial organization of these hidden states, Cloud-ScPO can derive preference signals without needing fully verified answers or external reward models. Experiments on GSM8K and MATH-Numeric datasets demonstrated significant improvements over existing ScPO methods, with gains of up to 4.49% on GSM8K. AI

IMPACT This research could lead to more efficient and effective training of LLMs for complex reasoning tasks by reducing reliance on extensive labeled data.

RANK_REASON The cluster contains a research paper detailing a new method for LLM reasoning. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• alphaXiv
• arXiv
• CatalyzeX
• Cloud-ScPO
• DagsHub
• Gotit.pub
• GSM8K
• Hugging Face
• MATH-Numeric
• ScienceCast
• SCPO

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →