LLM reasoning exhibits irrationality beyond value alignment, study finds

A new research paper from arXiv explores the concept of "rational value risk" in large language models, suggesting that even well-aligned models can exhibit irrationality during reasoning. This risk is quantified as a discrepancy between a model's deployed reasoning strategy and one that would rationally maximize aligned utility. The study, which tested models including Llama-3.1, Qwen-2.5, Tülu-3, GPT-5.2, GPT-5.5, and DeepSeek-V4 across various benchmarks, found this risk to be widespread. While value alignment can reduce, it cannot eliminate this irrationality, though techniques like self-consistency and longer chains of thought can improve rationality. AI

IMPACT Highlights a potential limitation in LLM reasoning that persists even with value alignment, suggesting ongoing challenges in achieving truly rational AI behavior.

RANK_REASON Research paper published on arXiv detailing a new concept in LLM reasoning. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• AlpacaEval
• arXiv
• DeepSeek-V4
• Fengxiang He
• GPT-5.2
• GPT-5.5
• GSM8K
• HumanEval
• Llama-3.1
• MathArena
• Qwen-2.5
• Tülu-3
• UltraFeedback

• paper
• safety

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →