Transformers Execute Causal Structure Learning Algorithms

Researchers have developed a novel approach to causal structure learning by utilizing linear-attention transformers. This method involves constructing a fixed-weight transformer that precisely replicates one update step of a standard continuous causal discovery algorithm. The transformer carries the current causal graph and the algorithm's multiplier between updates, which is crucial for exact execution. Experiments demonstrate that the constructed transformer block accurately matches a reference update, and when applied to synthetic data and benchmark network topologies, it inherits the successes and failures of the reference solver, highlighting the distinction between accurate algorithm execution and accurate causal recovery. AI

IMPACT This research demonstrates a new method for applying transformer architectures to complex algorithmic tasks like causal discovery, potentially expanding their utility beyond traditional NLP and sequence modeling.

RANK_REASON Research paper detailing a novel application of transformers for causal structure learning. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• alphaXiv
• arXiv
• CatalyzeX Code Finder for Papers
• CORE Recommender
• DagsHub
• Gotit.pub
• Hugging Face
• IArxiv Recommender
• Influence Flower
• Linear-Attention Transformers
• ScienceCast
• transformers

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →