New framework DATG diagnoses multilingual math reasoning flaws in LLMs

Researchers have developed a new framework called Directed Acyclic Trace Graphs (DATG) to diagnose issues in large reasoning models (LRMs) when performing mathematical reasoning in languages other than English. The study found that LRMs struggle not only with understanding non-English problem statements but also with the reasoning execution itself when the model's internal reasoning language is not English. Experiments using the Qwen3 series across 12 languages revealed that non-English reasoning often leads to reduced coverage of mathematical concepts and weaker dependency fidelity, particularly in low-resource languages. To address these identified failure modes, the researchers proposed two test-time controls, Loop-Retry and Formula-Retry, which demonstrated consistent improvements in target-language reasoning performance. AI

IMPACT Introduces a diagnostic tool to improve multilingual mathematical reasoning in LLMs, potentially enhancing their reliability in diverse linguistic contexts.

RANK_REASON Academic paper introducing a new framework and diagnostic methods for LLM reasoning. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• Directed Acyclic Trace Graphs
• Formula-Retry
• Loop-Retry
• Qwen3

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →