Medical LLM failures are decodable but uncorrectable by linear steering

Researchers have identified a phenomenon in medical large language models called Overthinking (OT), where models answer correctly in standard QA but fail in extended chain-of-thought reasoning. This failure state is linearly decodable with high accuracy, yet attempts to correct it using fixed linear steering methods proved ineffective across different architectures and domains. The study suggests that the failure signals are entangled with critical task computations, hindering direct correction but enabling improved post-generation reliability estimation. AI

Summary written by gemini-2.5-flash-lite from 1 source. How we write summaries →

IMPACT Identifies a specific failure mode in LLMs that hinders correction but aids reliability estimation.

RANK_REASON Academic paper detailing a specific failure mode in LLMs and exploring correction methods. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• Overthinking
• LLM
• Qwen2.5-7B
• MMLU-STEM

• paper
• safety