Researchers have introduced YFPO (Yoked Feature Preference Optimization), a novel framework designed to enhance the reasoning capabilities of large language models. This method couples response-level preference learning with neuron-level rewards by identifying math-associated internal features using AttnLRP. An auxiliary reward is derived from the activation margin of these neurons between preferred and dispreferred responses, which is then combined with the standard preference optimization objective. Initial experiments on the GSM8K dataset with a compact language model indicate that these neuron-guided rewards can positively influence optimization dynamics and lead to measurable improvements in reasoning tasks. AI
IMPACT This research could lead to more capable LLMs for mathematical reasoning by leveraging internal model states.
RANK_REASON The cluster describes a new research paper detailing a novel method for improving LLM reasoning. [lever_c_demoted from research: ic=1 ai=1.0]
- arXiv
- AttnLRP
- GSM8K
- Hugging Face
- Neuron-Guided Rewards
- YFPO
- Yifan Le
- Yoked Feature Preference Optimization
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →