Researchers have introduced CRoS, a new metric for evaluating the causality of Chain-of-Thought (CoT) reasoning in language models. Unlike previous methods that perturb reasoning steps, CRoS scrambles the order of these steps without altering their content, aiming to better isolate true causality. This approach is designed to prevent issues like reduced informativeness or self-correction that can arise from outcome-based rewards. In evaluations on the Reasoning Gym using the Gemma 3-4B model, CRoS demonstrated superior performance in accuracy, CoT quality, and faithfulness compared to existing causality and outcome rewards. AI
IMPACT Improves evaluation of LLM reasoning, potentially leading to more reliable and faithful AI systems.
RANK_REASON Academic paper introducing a novel method for evaluating LLM reasoning. [lever_c_demoted from research: ic=1 ai=1.0]
- Causality Reward on Scrambled CoTs
- CoT-Pass@K
- Gemma 3-4B
- Reasoning Gym
- Reinforcement Learning with Verifiable Rewards
- RLVR
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
