Plain Transformer model PENCIL outperforms GNNs in graph link prediction

Researchers have developed PENCIL, a plain Transformer model that can predict links in large graphs more efficiently than traditional Graph Neural Networks (GNNs). Unlike existing Graph Transformers that require complex structural encodings, PENCIL uses attention over sampled local subgraphs. This approach allows it to capture rich topological dependencies and outperform GNNs in parameter efficiency and scalability, even without node features. AI

IMPACT Demonstrates that simpler Transformer architectures can achieve competitive performance in graph machine learning, potentially reducing the need for complex GNN engineering.

RANK_REASON Academic paper introducing a new model architecture for graph link prediction. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• arXiv
• Graph Neural Networks
• PENCIL
• Transformer

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →