New study re-evaluates causal direction methods, finds inflated results

A new paper titled "One Ruler" re-evaluates causal direction methods on the Tuebingen dataset, arguing that current comparisons are flawed due to differing protocols. The authors conducted a "same-hands" re-evaluation, applying all methods to the identical 102 pairs without parameter tuning. They introduced a parameter-free baseline using sorted-conditional compression, which achieved competitive results, highlighting issues like test-set model selection and significance-gated abstention that inflate published figures. AI

IMPACT This research highlights potential inflation in published results for causal direction methods, suggesting a need for standardized evaluation protocols in AI research.

RANK_REASON The cluster contains an academic paper detailing a new evaluation methodology and baseline for causal direction research. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• One Ruler
• SLOPE

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →