Graph Transformers for MILPs Limited by 1-WL Test, Study Finds

A new paper characterizes the expressive power of global-attention graph transformers used for mixed-integer linear programs (MILPs). The research proves that these models, including architectures like Graphormer and Set Transformer, are limited by the one-dimensional Weisfeiler-Leman (1-WL) test. This means that MILP instances which are equivalent under the 1-WL test will receive identical graph embeddings from these transformers, preventing the recovery of certain graph invariants. The study suggests that expressiveness beyond 1-WL primarily comes from input encoding rather than the attention mechanisms themselves. AI

IMPACT Characterizes the expressive limits of graph transformers, potentially guiding future model development for complex structured data.

RANK_REASON Academic paper published on arXiv detailing theoretical limitations of graph transformer models. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• arXiv
• Gasse
• Graph Foundation Models
• GraphGPS
• graph neural networks
• Graphormer
• Graph Transformers
• Mixed-Integer Linear Programs
• Set Transformer
• Weisfeiler–Leman algorithm

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →